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FOREWORD 


This book is the second by the authors relating to Teleoperators 
and Human Augmentation. It is a survey of an emerging facet of 
control technology which, when adopted by industry, will have a 
profound effect upon the development of new products and new 
processes. The technology of sophisticated control — particularly the 
cybernetic interaction between man, computer and versatile 
machines — is important in the fields of nuclear energy, in our 
investigation and use of the ocean’s resources and in space exploration. 
Further applications of teleoperator systems will occur in industry, 
urban services, and even agriculture. Teleoperator control is becoming 
an essential part of our scientific-engineering-industrial basis for a more 
advanced society, one in which knowledge and technology — rather 
than nature and man — can be exploited. 

The sources of teleoperator technology are found in recent work in 
biomechanics, computer science, and the remotely operated equipment 
used by the AEC and NASA. Teleoperator systems augment and extend 
man — they do not replace man. Man is always in the control loop, a 
characteristic that distinguishes the teleoperator system from automa- 
tion. 

It is hoped that this book, describing the current development of 
teleoperator control theory and hardware, will stimulate further 
refinements in this fascinating symbiosis between man and machine. 
New approaches are needed if society is to fully realize the benefits of 
modern science. The virtually unlimited energy made available by 
nuclear science, the new level of sophistication being achieved by our 
chemistry and physics, our earth sciences and life sciences — all this 
coupled with the ability to augment man through teleoperators, will 
hopefully propel us toward a society in which man can at last have time 
and resources to think and act in new directions and new dimensions. 



Glenn T. Seaborg 
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Chapter 1 


INTRODUCTION 


A teieoperator is a general-purpose, dexterous, man-machine system that 
augments man by projecting his manipulatory and pedipulatory capabilities 
across distance and through physical barriers into hostile environments. A 
manipulator that fabricates radioactive fuel in a hot cell is a teieoperator 
because it transmits man’s dexterity through the hot-cell wall into the lethal 
environment. Teleoperators can also extend man’s hands to the Moon and 
planets before he can make the trip himself (Table 1). An artificial limb is 
considered a teieoperator because it augments an amputee by restoring a part 
of his lost dexterity — the lost limb is in effect a barrier to normal operation. 

Man is nearly always in the teieoperator control loop. In contrast, 
clock-radios and computer-controlled machine tools are preprogrammed 
machines, not tele operators, because man is never in the loop on a real-time 
basis. Neither are robots tele operators. Robots operate autonomously and 
sometimes (in science fiction, at least) counter to man’s interests. The 
man-machine relationship in the teieoperator is essentially symbiotic ; that is, 
mutually beneficial. Man needs the machine’s strength and resistance to 
hostile environments, while the machine depends upon man’s brain and 
dexterity. 

The adjective “dexterous” in the definition of the teieoperator excludes 
the great host of nondexterous man-machine systems that surround us. To 
illustrate, the automobile becomes a part of a man-machine system when the 
driver brings it to life with the ignition key; yet the automobile cannot 
manipulate anything for all its other attributes. 

In metal-working plants or construction jobs, and on many production 
lines, one sees man-machine systems that pick up, position, and otherwise 
manipulate objects and materials. We prefer to exclude these single-purpose 
mechanical hands and fingers from the class of teleoperators by adding the 
adjective “general-purpose” to the definition. Teleoperators should be 
thought of as extenders and augmenters of man; and man is a general- 
purpose, versatile creation. 

SCOPE OF THIS SURVEY 

In a previous survey, 1 we have described the ascendancy of teleoperators 
and how they are already expanding man’s environmental frontiers as his 


1 



to 


Teleoperator Type 


Manipulators (Fig. 1) 


Prosthetic and orthotic 
devices (Fig. 2) 

Man amplifiers 
(Fig. 3) 


Walking machines 
(Fig 4) 


Table 1 The Four Major Classes of Teleoperators 


Characteristics 


Mechanical analogs* of human arm and hand. May be 
fixed or attached to vehicles. 

Mechanical analogs* of human arm and hand, attached 
directly to the body. 

Mechanical analogs* of entire or a large portion of 
the human body. Generally, these are exoskeletal 
machines that fit around the body somewhat like 
a knight’s armor. 

Mechanical analogs* of human legs controlled by 
operator directly (not preprogrammed). 


Typical Applications 


Common in radioactive hot cells, undersea research 
craft, and similar “hostile” environments. Many 
thousands now in use. 

Artificial limbs replace the natural limb, while 
orthotic devices aid damaged or weakened members. 

In common use. 

Handling heavy loads, particularly in military and 
undersea environments. Still in development stage. 

Locomotion over rough terrain unsuited to wheels. Non- 
preprogrammed types still being developed. 


*The analogs are generally not exact. They usually have many fewer degrees of freedom than a human; although they often incorporate degrees of 
freedom unknown to humans, such as joint extension. 
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Figure 3 Artist's sketch of the General Electric /Department of Defense 
powered man amplifier (Hardiman I). (Courtesy of R . S. Mosher, General 
Electric Co.) 



Figure 4 Artist's concept of the General Electric /Department of Defense 
Walking Truck . (Courtesy of General Electric Co.) 
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proxy and precursor. In this survey, we focus on the subject of teleoperator 
control. 

Flexible, y responsive, sensitive control is a keystone of successful 
tele operator design. But because man and machine are linked so intimately in 
the teleoperator, teleoperator control engineering is not as well developed as 
it is, say, in missile control, where man is usually not in the loop. Man with 
his strengths, his weaknesses, his nonlinearity, and his manifest complexity 
makes teleoperator control a challenging field. 

No well-defined body of theory and practice bears the label “teleoperator 
control.” The lore and literature of teleoperator control consist of bits and 
pieces from many disciplines. The Atomic Energy Commission (AEC) has 
pioneered the control technology of mechanical and electrical master-slave 
manipulators,* especially at its Argonne National Laboratory (ANL). The 
National Aeronautics and Space Administration (NASA) has contributed to 
teleoperator control in two quite different ways: (1) the theory of manual 
control as used in describing the piloting of aircraft; and (2) the use of 
computers as aids to human operators in situations where significant time 
delay exists (predictive or preview control) and where control tasks are 
complex (supervisory control). The Department of Defense (DOD), like 
NASA, has studied manual control in depth in connection with aircraft. It has 
also sponsored considerable research and development work on walking 
machines and man amplifiers. Further, in connection with M.I.T.’s Project 
MAC (Machine-Aided Cognition), DOD has investigated computer-controlled 
manipulators. To the government-sponsored work must be added centuries of 
hardware developments in the fields of prosthetics and orthotics, where 
practical techniques have been emphasized more than theory. Assimilating 
these bits and pieces, we can construct a comprehensive though incomplete 
description of teleoperator control. This integration is a major objective of 
this survey. 

The titles of Chapters 3 through 6 — the cornerstone technical chap- 
ters — may be paraphrased as: control theory, the man-machine interface, 
control hardware, and displays. Only the chapter on displays is deliberately 
restricted; the subject is so broad that we can deal only with those aspects 
that relate to teleoperator control. 


RESUME OF TELEOPERATOR DESIGN PRINCIPLES 

From hot-cell master-slave to walking machine, teleoperators vary widely 
in physical configuration. Nevertheless, generalization is possible: Fig. 5 
illustrates how a teleoperator can be disassembled into ten subsystems; all 
united by the mutual interchange of control signals, power, physical 


*See Table 3, Chap. 2, for manipulator definitions. 
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Figure 5 Teleoperator subsystems , showing some of the important inter- 
faces . S - spatial, E = electrical power , ME = mechanical, T = thermal, 
I ~ information, R = radiative, EM = electromagnetic. Dotted lines indicate 
local control loops that bypass the central control subsystem; viz., thermostat 
thermal control 


structure, and other “forces” at their interfaces. Except for the control 
subsystem, which will be covered in depth in this book, we now summarize 
the major facets of subsystem design as detailed in Ref. 1. 

The Actuator Subsystem. This subsystem carries out the manipulations, 
walking motions, and other dexterous activities ordered by the human 
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Figure 6 Installation diagram for the ANL E3 electric master slave. Motions 
of the master control arm and hand are communicated to the slave-side 
actuators electronically . (Courtesy of Argonne National Laboratory.) 


operator. The actuator subsystem is the effector portion of the teleoperator. 
The slave arms and hands of the familiar master-slave manipulators comprise a 
typical actuator subsystem. The purely mechanical manipulators, where man 
activates the slave arms and hands with cables somewhat analogous to puppet 
strings, are the most common. Electrohydraulic actuators are favored for 
underseas work and in walking machines and man amplifiers because of their 
strength per unit volume. ANL has built several varieties of electrical 
master-slaves for nuclear work (Fig. 6). Many switch-controlled electric 
manipulators are now in service in many fields (Fig. 7). These differ from the 
electric master-slaves in that force feedback is not present. 

The Sensor Subsystem. The sentient portion of the tele operator may 
see, feel, hear, or otherwise sense the environment, giving the operator 
rapport with transactions in the working area. More than any other 
subsystem, the sensors enable man to project himself across distance and 
through barriers to the spot where his appendages are partially duplicated in 
metal, plastic, and other inorganic media. Television cameras and direct 
viewing are common in teleoperator work because sight is almost always the 
most useful of man’s senses in manipulation. Force feedback and tactual 
sensation are the next most useful, particularly in assembly of tight-fitting 
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Figure 7 A typical switch-controlled electric unilateral manipulator. 
(Courtesy of R. Karine n, Programmed and Remote Systems.) 

parts and similar operations. Some force feedback exists in mechanically 
connected manipulators, such as the omnipresent mechanical master- slaves, 
where cables transmit forces both to and from the operator. The ANL 
electrical master-slaves and some electrohydraulic tele operators employ servos 
to provide force feedback. Looking to other possibilities, microphone 
pickups, sonars, infrared cells, and gyros are only a few ways in which 
information about the working area can be transmitted back to the operator. 
Some sensors are nonanthropomorphic — Moire-pattern tactual sensors, for 
example — and the operator has to learn how to interpret this unnatural 
feedback. In the control subsystem, all kinds of feedback are melded into a 
presentation or display (not necessarily visual) that helps the operator 
identify himself with the task and its environment. 
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The Communication Subsystem. Wires, electromagnetic links, and me- 
chanical linkages, form the nervous system of the teleoperator. Feedback 
information from the sensor subsystem speeds back to the operator via the 
communication subsystem. Likewise, his commands are transmitted in the 
opposite direction to the actuator subsystem. In the simple switch-controlled 
unilateral manipulators, only a narrow bandwidth is needed to transmit the 
operator’s commands. At the other extreme, television feedback requires a 
very wide bandwidth. At a more primitive level, the two communication links 
in a mechanical master-slave are those created by the push and pull of the 
control linkages and direct viewing of the target through, say, a hot-cell 
window. 

The Computer Subsystem. Man often needs help in controlling a tele- 
operator. Signal time delay, task complexity, and human response time bring 
computers into the picture. Computers can make calculations and predictions 
that improve the operator’s decision making capability. Computers can also 
operate using fast, sophisticated subroutines that relieve the operator of some 
of the routine operations, such as the stowing of a submersible’s manipulator. 
In more advanced teleoperators, computers will be able to generate displays 
from the incoming feedback data. When we discuss teleoperator computers, 
we are generally talking about the future. While digital computers have been 
studied for space and undersea applications, the only operational teleoperator 
computer is an analog computer employed by the Navy’s Deep Sea Rescue 
Vehicle (DSRV) for controlling its manipulator arm during a few routine 
operations. In the realm of experiment, NASA and DOD have supported 
research on manipulators controlled by digital computers at M.I.T., Case 
Western Reserve, and elsewhere. 

The Propulsion Subsystem. Motor-driven wheels, rockets, screws, and 
the leg-like parts of exoskeletons serve to propel teleoperators from place to 
place. Rockets and gas-jet propulsion units have been proposed for orbital 
operations utilizing tele operators. Beneath the ocean, screws and water jets 
are already used for research and military submersibles carrying manipulators. 
On land, a variety of wheeled and tracked manipulator-equipped vehicles have 
been constructed for nuclear rescue and cleanup work. 

The Power Subsystem. Man himself provides the power in mechanical 
master-slaves and in some artificial limbs. Wherever commercial power lines 
go, they are a ready source of power. In space and underseas, power sources 
usually have to be carried along with the teleoperator. Chemical units (fuel 
cells and internal combustion engines), batteries, and nuclear power sources 
have been suggested. In general, the teleoperator operates from the power 
supply of the vehicle or building where it is attached. The so-called externally 
powered artificial limb is an exception; it requires a special battery or source 
of compressed gas. 
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The Attitude-Control Subsystem. To stabilize and maintain the spatial 
orientafioil of a teleoperator, particularly of space vehicles, submersibles, and 
walking machines, a variety of jets, screws, docking arms, and balancing 
devices has been designed. In space and under the sea, where vehicles “float” 
freely, reaction engines are almost essential, although magnetic attitude 
control subsystems acting in conjunction with the Earth’s field are feasible 
for small Earth satellites. Walking machines and man amplifiers with legs, of 
course, use their feet for maintaining balance and an upright position. The 
attitude control subsystem also includes the sensors that measure attitude, 
such as horizon sensors, gyroscopes, and even man himself through his sense 
of balance. 

The Environment Control Subsystem. This subsystem maintains proper 
temperatures, atmospheres, radiation levels, etc. within the teleoperator. 
Radiators would be employed for dissipating excess heat in outer space; in 
marine applications, seawater is an excellent heat sink. Vehicle atmospheres 
for astronauts and their watery counterparts are either carried along and 
expended, or recycled. Shields protect the teleoperator and its operators from 
radiation (from space or a nuclear power plant) and meteoroids. 

The Structure Subsystem. The structure subsystem unites and supports 
the other subsystems. In teleoperators, of course, the system-as-a- whole is 
often divided physically by an environmental barrier or by great distances. 
When the operator is safe on Earth or on a surface ship, the manipulator- 
carrying vehicles may be open to vacuum, seawater pressure, radiation, 
smoke, or whatever constitutes the hazardous environment. When man goes 
along, he must be protected by space-capsule walls, massive hulls, and the 
like. Many of the mobile, terrestrial teleoperators are simply built like tanks, 
trucks, or bulldozers. The structures of the teleoperator arms and legs 
themselves usually mimic the human body in form but generally employ 
external rather than internal skeletons. Man amplifiers, for example, are 
patterned after the insect world because they fit around the human operator 
like a beetle’s hard carapace. 

In the structure subsystem we see best the many facets of this class of 
man-machine systems we call tele operators. The actuator terminals mimic 
man, but the bodies are heavy shells or skeleton-like: they are wheeled or 
many-legged, hung on hot-cell walls, or mounted on tank bodies. At the 
operator-machine interface, we again find great variety, for man’s senses are 
manifold and there are a multitude of ways in which he can bring life to his 
ersatz hands and arms, be they on the Moon or in some dark, abyssal ocean 
trench. The rest of this book describes how man controls this most versatile 
kind of machine. 



Chapter 2 


A SKETCH OF THE CONTROL SUBSYSTEM 

ESSENCE OF CONTROL 

To control his machines man acts primarily as a goal-setter and an error 
corrector. He decides what he wants the machine to do; he plans the strategy; 
he gages the machine’s deviation from desired performance; and he 
manipulates the machine’s controls in a way that reduces the error. He does 
this when he steers a car along a winding road and when he picks up a sample 
of lunar soil with a tele operator hand from a distance of a quarter-million 
miles. These words and the control schematic shown in Fig. 8 actually 
oversimplify the situation. Any control system that counts a human being 



Figure 8 The teleoperator control loop (shaded), with computer support for 
supervisory control, display generation, and direct support of the operator . 

among its elements is complex and difficult to describe scientifically because 
man himself is so complex and difficult to describe. 

Why, then, admit man to the tele operator control loop? Machines can 
certainly detect their own errors and correct them; autopilots and home 
heating plants do this very nicely. The reason for man’s presence stems from 
his ability to set strategy and to deal with the unexpected — those situations 
we cannot preprogram into a machine’s memory. Man is an adaptive creature; 
and, if tele operators are to be the extension of man, they must be adaptive 
also. To illustrate: Could a pure machine uncomplicated by man’s presence 
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figure out how to repair a ruptured oil pipeline far out on the continental 
shelf? 

In principle, the answer to the foregoing question is “yes.” Adaptive 
machines, machines that learn from experience, can and have been built. 
They are true robots. Today’s robots, however, cannot approach man’s 
adaptability, versatility, and intelligence. It would take many ruptured 
pipelines before a robot learned how to fix them. For decades, at least, 
man-operated tele operators will reign supreme in those hazardous and distant 
spots where man prefers to send machine proxies. 

Most extant teleoperators are “pure” man-machine systems; that is, man is 
always in the control loop. As teleoperator technology progresses, though, 
preprogrammed subroutines are being added to relieve the operator of those 
wearisome, repetitive tasks that can be done better by machines. A very 
simple and basic preprogrammed subroutine is one which stops tele operator 
arm motion when limit switches indicate that self-inflicted damage is 
imminent. Most complex machines include similar localized reflex control 
arcs that intrinsically react faster than man. Subroutines are also extremely 
useful in space operations — say, lunar exploration — where there is signifi- 
cant signal time delay between operator and teleoperator hands. Such 
subroutines, which are intrinsic to Sheridan’s and Ferrell’s supervisory control 
approach , 2 do not add to a teleoperator’s intelligence or adaptability, but 
they improve overall effectiveness considerably, especially where time delays 
are large. In principle, then, a continuous spectrum of teleoperators exists be- 
tween the pure, man-always-in-the-loop extreme to the completely prepro- 
grammed, dexterous, general-purpose machine possessing only an ON-OFF 
switch, in other words, a robot . As technology progresses, we may expect to 
see teleoperators move toward the robot end of the spectrum. 

The ingenuity of man and his passion for making machines that emulate 
himself should not be discounted. The future may soon see the addition of 
adaptive or artificially intelligent subroutines to teleoperators. At first, some 
of the simpler, more routine decisions might be machine-made. Eventually, 
both preprogrammed and adaptive subroutines might be added until man 
could say to a machine, “Go and explore the galaxy for me.” Philosophically 
speaking, the tele operator may be a transitional man-machine system that 
presages generations of machines that are man-like, man-directed, man- 
seiving, and yet self sufficient save for a few spoken commands from their 
masters. This would be the ultimate man-machine world that Landers calls 
the dybosphere . 3 

The moral behind these projections of what may come (and sooner than 
we expect) is that the relatively crude switch and servo controls of 
contemporary teleoperators are merely harbingers of a man-machine partner- 
ship that will be effected by the spoken word, the gesture, and the nuances 
that command the best servants. 
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MAKEUP OF THE CONTROL SUBSYSTEM 

At the core of the control subsystem is the human operator (Fig. 9). 
Toward him flow feedback data that describe the positions and velocities of 
the teleoperator’s hands, arms, and other actuators as well as the objects 
being manipulated. From him flow the commands that will reduce (hope- 
fully) the error he perceives in teleoperator performance. The human brain is 


CONTROL 

SUBSYSTEM 



Figure 9 The man-machine interface is penetrated primarily by display data 
and commands , but man also interacts with almost every subsystem in some 
way. 
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the goal-setter and the error computer, planner, and decision maker, although 
a computer may supplement some brain functions. 

Two man-machine interfaces are of paramount importance. First, 
feedback information from the machine part of the tele operator must be 
“read into” the brain so that a performance error can be computed. Current 
terminology calls the device that translates machine sensor readings into 
signals comprehensible to the brain a display (Chap. 6). A display may be 
simply a faithful television view of the scene or it may be a symbolic display, 
such as a meter indicating the grip force exerted by the teleoperator hand. 
The second critical interface separates man from the teleoperator actuators, 
as well as other teleoperator subsystems under the operator’s direct control. 
Man’s commands to his machine partner stream through his central nervous 
system to his arms, hands, eyes, tongue, and other parts of his anatomy that 
can create mechanical, sonic, and electrical signals. These signals cross the 
man-machine interface and activate controls that convert them into com- 
mands comprehensible to the rest of the tele operator (Chap. 5). 

The complete circuit from man to machine and back to man is the 
control loop. Information courses around this loop, which may be augmented 
by computers here and there. The successful operation of the teleoperator 
depends upon the successful encoding, transmission, and translation of this 
data stream. 

Perhaps this portrait of the teleoperator control subsystem seems overly 
formal for the simple mechanical master-slave manipulator where the 
operator sees his task through a window (direct visual feedback), but it is 
none too rigorous and precise for the sophisticated teleoperators of the future 
which must work on distant planets, on the deep sea floor, and in other 
hazardous environments that man cannot penetrate safely. 


MAN AS AN ELEMENT IN THE CONTROL SUBSYSTEM 

The human operator eludes precise definition. If he did not, control 
engineers could formulate an elegant human transfer function or human 
describing function that would mathematically describe what man would do 
when confronted with feedback data and decisions to make. The human 
transfer function describes what a normal man will do given a specific input. 
In the next chapter, we will describe some of the human transfer functions 
that have been synthesized for extremely limited situations. Unfortunately, 
they have scant utility in teleoperator control theory, except for helping 
predict system stability and in very special situations. In tele operators as 
nowhere else, man is an adaptable, rather unpredictable element that cannot 
be encompassed by formulas. 

In lieu of precise mathematical human describing functions, words must 
suffice. It is common to describe man in terms of his input-output 
characteristics, just as if he were an electronic control component or black 
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box . 4 The words, however, can only guide us in the design of the tele operator 
control subsystem. 

The sensory input channels leading to man’s brain are many. We know 
how to use them but not how or why they work as they do. From this wide 
selection, only four of our senses are in actual use today in teleoperator work; 
vision, audition, and the cutaneous and kinesthetic senses; i.e., sight, sound, 
touch, and the sense of position and motion (Table 2). 


Table 2 Some Input Channels of Lesser 
Importance to Teleoperator Design* 


Sense 

Sense Organ (s) 

Rotation 

Semicircular canals 

Falling, rectilinear motion 

Muscles, semicircular canals, 
otoliths 

Taste 

Tongue, mouth 

Smell 

Nose 

Vibration, pressure 

Skin, underlying tissue 

Temperature 

Skin, underlying tissue 

Attitude, balance 

Semicircular canals, otoliths 

Bodily motion, position 

Joints, tendons 

Passage of time 

9 


*See tables 5, 6, and 7, Chap. 4, for a detailed description 
of the characteristics of man’s senses of vision, audition, etc. 


Sight is by far the broadest channel carrying feedback information to the 
operator. In fact, it is the only input channel in most teleoperator systems. 
This is true because sight is practically indispensable* in manipulatory 
tasks — we have to have it — and visual channels are relatively easy to build 
(windows, TV, etc.). Force feedback is present in mechanical and electrical 
master-slaves as well as some walking machines and man amplifiers currently 
under development. Proprioceptive feedback or sense of limb position can be 
achieved by using exoskeletal controls that maintain the same configuration 
as the actuators. The man amplifier in Fig. 3 possesses such exoskeletal 
controls. Touch sensation, as opposed to gross force feedback, is highly 
desirable in a teleoperator but sometimes not worth the cost of instrumenta- 
tion; it has not been developed to the point where it is used regularly. Sound 
waves coming from manipulatory processes carry alarm or warning signals 
(viz., a dropped object); and for this reason a few manipulators incorporate 
microphones. 

Despite the paucity of feedback channels in contemporary tele operators, 
designers always have as their ultimate goal the faithful reproduction 


Manipulation by force feedback alone is possible but it is generally not efficient 
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(occasionally, amplification) of most of the sensations that would normally 
be experienced by an unaided human actually doing the job of the 
teleoperator. In practice, they settle for much less. Of course, no one 
reproduces all aspects of a hazardous environment for the operator — just 
those aspects of the environment that will aid manipulation. For example, the 
forces experienced by the machine body of a man amplifier would crush the 
human operator if they were not attenuated. 

Humans also have subtle input problems. For example, the all-important 
visual channel is subject to a great variety of optical illusions and signal 
distortion. Then, there is operator fatigue which can seriously distort the 
information presented to the decision-making and command-generating 
portion of the brain. Fatigue also lengthens the operator’s reaction time. 
Finally, man’s senses are far from the easy-to-analyze linear transducers that 
engineers like so much; that is, the intensity of a stimulus perceived by the 
operator is not proportional to the actual physical magnitude of the stimulus. 
Instead, each sensory channel seems to exhibit a different power law 
relationship.* 

To illustrate the complexity of the problem, some evidence suggests that, 
if a system has anthropomorphic features, the operator instinctively employs 
his long-used anthropomorphic responses. This may be undesirable if the task 
or feedback is nonanthropomorphic. Yet, in hot-cell work the roughly 
anthropomorphic master-slaves have proven to be highly effective. 

Human weaknesses are counterbalanced by unexpected strengths that 
transcend the usual adaptive and integrative powers. Airplane pilots, 
astronauts, and other operators of complex machines show a surprising ability 
to handle nonanthropomorphic displays and manipulate controls that 
certainly seem “unnatural.” In fact, man may overpamper himself and 
unnecessarily restrict the machine by making his teleoperators too much after 
the human mold; although some engineers object to this contention. 

The gist of this discussion of input channels is that the human operator 
may confound control theoreticians with his nonlinearities and unpredict- 
ability but that he also possesses useful properties that no artificial brain can 
yet match. 

Once the human operator has digested the stream of input information 
and decided upon a course of action, he “emits” a train of command or 
output signals. Precisely what transpires between input and output in the 
human transducer has been argued by speculative philosophers for centuries. 
In other words, we really have little idea of how information is processed in 
the brain; and for practical purposes we do not really need to know. 


* There is, however, a simple and fairly well substantiated law (Weber’s Law) that 
states that for each sensory channel the ratio between the just noticeable difference in 
the perceived stimulus and the actual physical magnitude of the stimulus remains 
constant More sophisticated laws also exist 4 
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To translate his commands into machine language, the operator has at his 
disposal his hands, feet, head, in fact any part of his body that moves, even 
his eyeballs /(Chap. 6). By far the most useful output channel depends upon 
the motion of the human hands. In current teleoperator design, the 
preponderance of hand-actuated controls is even more marked because 
manipulators are really machine analogs of man; and it seems eminently 
logical to control hands with hands. Similarly, in a biped walking machine it 
is natural to control legs with legs. 

When the tele operator must be steered or flown, or it possesses more 
degrees of freedom than the operator can handle with his hands and feet, the 
human voice may serve as an output channel. Even today, machines can be 
designed to recognize a small array of spoken commands, such as “turn left” 
or “stop.” 

Suppose a handicapped person has no hands or arms to control his 
artificial limbs. Muscle-bulge switches and shoe switches are sometimes 
employed. More often, limb remnants and shoulder muscles activate 
prostheses. A promising human output channel, still in the research and 
development stage, translates the weak electrical signals created within the 
body by muscle action into electrical commands a machine can understand. 
Muscle action potentials (MAPs) form the basis of electromyographic (EMG) 
control of artificial limbs as well as other types of teleoperators (Fig. 10). 



Figure 10 An orthotic arm controlled by electromyographic (EMG) signals 
generated by the amputee's muscles. (Courtesy of Case Western Reserve.) 
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SOME SPECIAL TELEOPERATOR CONTROL PROBLEMS 

Most treatises on the human control of machines— manual control , as the 
dicipline is often called — deal with aircraft, terrestrial vehicles, and other 
machines with far fewer human characteristics than teleoperators. Because 
teleoperators simulate human traits, one would expect that matching the man 
and machine portions would be easy (Fig. 9). It is actually a most difficult 
task, one requiring for discussion a full chapter later (Chap. 4). Here, we 
merely list some of the more troublesome aspects of man-machine integration 
to illustrate how teleoperator control is different. 

1. The operator is often located at a point far removed from the 
mechanical arms and hands he is controlling. In most terrestrial 
hot-cells, where good visual displays and force feedback exist, it is not 
too difficult for the operator to project himself into the working area; 
that is, identify his movements with those of the distant hands and 
arms. The problem here is the provision of good feedback in more 
difficult applications, such as undersea manipulation. 

2. Great distances between the operator and the actuator subsystem 
introduce signal time delays that confuse the operator. This problem is 
serious in some space applications; viz., the round-trip signal transmis- 
sion time to the Moon is about 2.6 sec. 

3. A sophisticated teleoperator has so many degrees of freedom (over a 
dozen in many, instances) that an operator is hard put to control them 
in concert unless both controls and actuators possess some anthropo- 
morphic characteristics (Fig. 11). 

4. If the motions and dimensions of the mechanical hands and arms do 
not correspond rather closely to the motions of the controls, operator 
confusion may result. For example, moving a control left should cause 
the appropriate actuator to move left. (See Table 3 for definitions of 
spatial and visual correspondence.) 

As we shall see, man-machine integration difficulties such as these can be 
solved through the application of good human engineering , a subject that now 
includes the application of computer aids. 


PERFORMANCE FACTORS 

A cornerstone of systems analysis is the formulation of an overall figure 
of merit that describes the performance of weapons systems and other 
complex man-machine conglomerates in terms of a single parameter. The 
parameter “cost effectiveness” has achieved fame and some notoriety in 
many fields.. Teleoperators have no such advantage; perhaps they are more 
subtle than weapons systems. 




Figure 11 This often-printed photograph of the General Electric electrohydraulic Handyman illustrates 
the degree of coordination possible between master and slave arms in a teleoperator with force feedback . 

The arm-paralleling controls exhibit anthropomorphic characteristics . (Courtesy of R . S. Mosher , General 5 

Electric Co.) 
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Table 3 Some Definitions Used in Teleoperator Control 


Open loop 
Closed loop 


Preprogrammed 

Adaptive 

Robot 

Time delay 


Preview control 
Supervisory control 


Spatial cor- 
respondence 
Visual cor- 
respondence 
Degree of freedom 

Anthropomorphic 

Quickening 


Unilateral tele- 
operator 
Bilateral tele- 
operator 


Rectilinear tele- 
operator 


Master- slave 
teleoperator 


No feedback of any kind to operator 

Some kind of feedback is present Psychologists call a 
loop “closed” when vision is present but engineers 
usually reserve the term for nonvisual feedback. 

Commands are prerecorded 

Capable of making decisions based on past experience. 

An adaptive machine that needs no human operator, 
usually humanoid in form. 

Command and feedback delay due to: (1) signal trans- 
mission line; (2) coding delay; (3) passive-process delay 
(inertial effects); and (4) human reaction delay. 

Use of predictive displays (with time extrapolation) to 
help overcome the effects of time delay 

Use of computers at the operator end to aid decision making 
and at the actuator end for adaptive control and application 
of subroutines. 

Actuators mimic motion of controls (used primarily to de- 
scribe master- slaves and slaved TV systems). 

Visual display slaved to position of operator’s head 

A dimension of motion in a teleoperator; viz., wrist rota- 
tion and elbow pivot. 

Actuators or controls resemble human body segments in terms 
of degrees of freedom and how they are articulated. 

The use of time derivatives of teleoperator motion to help 
the operator predict actuator position and compensate for time 
delay. (A distant cousin of preview control.) 

A tele operator in which force and motion can be transmit- 
ted only from the operator controls to the actuators. 

A teleoperator in which force and motion can be transmitted 
from the operator controls to the actuators and vice versa; i.e., 
the slave arm can move the master arm. (Note: “bilateral” does 
not imply physical symmetry here as it does in biology.) 

A tele operator possessing several degrees of freedom in 
rectangular coordinates. Generally, these degrees of freedom are 
associated with over-head bridge-crane positioning systems. 
“Rectilinear” is often used incorrectly as a synonym for “uni- 
lateral.” Joints with angular freedom are often termed “polar” 
in the literature. 

A teleoperator in which forces and torques are proportionally 
reproduced from the controls (master) to the actuators (slave). 

A master-slave is bilateral in at least seven degrees of freedom 
in each arm/hand. All degrees of freedom can be controlled natu- 
rally and simultaneously. This term was originated at Argonne Na- 
tional Laboratory. 
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In experiments with manipulators, notably at the U. S. Air Force’s 
Aerospace Medical Research Laboratory, the time taken for a skilled operator 
to perform a manipulative task has been used as a gauge of merit. While useful 
in comparing different brands of manipulators, this parameter can hardly be 
expressed in terms oLengineering design variables, such as number of degrees 
of freedom or speed of joint rotation. Teleoperator designers usually rely 
upon a group of secondary figures of merit, which are collectively optimized 
by experience rather than systems analysis. We now list those secondary 
figures of merit related to the control subsystem. 


Figure of Merit 


Definitions, Comments, and Inter comparisons 


Torque, force, 
or grip 


Speed 

Accuracy 

Ease of indexing 

Articulateness 

Stiffness 

Friction 

Inertia 


Sponginess 


Applied to rotating joints and teleoperator 
hands. The control subsystem should be able to 
apply force and torque continuously or in grad- 
uated steps in response to the controls. Force 
multiplication between operator and actuator 
may be desired. Design levels depend upon task 
at hand. 

The linear or angular rate at which a joint 
moves. Related to torque, force, and the mass 
of the mechanical hands, arms, and legs. Speed 
should be controllable in many applications. 

An arm or hand is accurate if it responds to 
a command (say, rotate 30° clockwise) 
with some agreed-upon degree of precision. Pre- 
cise motion requires good controls. 

The ability of tele operator appendages to 
move into prescribed configuration. Computer sub- 
routines are sometimes used to index a tele- 
operator. 

A measure of the number of joints and degrees 
of freedom. Each degree of freedom complicates the 
control subsystem. 

A synonym for teleoperator rigidity. This is a desir- 
able quality (see sponginess). 

Energy dissipation during motion. This can 
tire the operator as well as degrade force feed- 
back. 

A measure of the difficulty of accelerating 
and decelerating the actuators beyond the time 
lags caused by circuitry, mechanical linkages, 
and signal transit time. Inertia can cause over- 
shooting and oscillations about a target posi- 
tion. 

A characteristic of pneumatic tele operators 
in which controls and actuators are connected 
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Backlash 


Stability 


Sensitivity 


Cross coupling 


Drift 


Compliance 


Reliability 


Fail-safe capability 


S elf-pro tectivity 
Cost 

Power requirement 


Support-equipment 

requirements 


Operator skill 
required 


by a compressible fluid. To some extent, good 
controls can eliminate sponginess (see stiff- 
ness ). 

The amount a control must be moved in the re- 
verse direction before the commanded joint re- 
sponds. 

The ability of a tele operator to move smoothly 
from one configuration to another and maintain 
it without jitter, hunting, or divergent oscil- 
lations. 

A tele operator is sensitive if a slight mo- 
tion of the controls causes actuator motion. 

Often “play” or a “deadband” will be built 
into the control subsystem to prevent excessive 
sensitivity. 

This occurs when commanded motion in one de- 
gree of freedom creates motion in another. The 
control subsystem design should preclude cross 
coupling. 

Drift occurs when electrically and hydrauli- 
cally actuated teleoperators may move slightly 
in a continuous fashion due to servo “leakage.” 

The match between the manipulatory require- 
ment of a task and the motion capabilities of 
the teleoperator (Fig. 11). Good control de- 
sign can improve the dynamic match. 

The probability that the system will operate 
at some stipulated level of performance for a 
stipulated length of time. The control subsystem 
must help the overall tele operator system meet 
reliability goals. 

When a tele op era tor fails or loses power, 
the control subsystem should assure that the 
actuators retain their configurations. (Collapse 
could be disastrous in a man- amplifier.) 

Limits switches and other control devices 
should prevent a tele operator from damaging itself. 

Self-explanatory 

Power is critical in space and undersea work. 

The control subsystem should draw as little power 
as possible. 

The total of all auxiliary equipment; such as 
repair and maintenance facilities, fuel-supply 
facilities and vehicles; and, of course, the trained 
technicians associated with this equipment. 

The effective matching of the man-machine inter- 
face can reduce skill requirements. 


f 



Chapter 3 


CONTROL THEORY 

OPEN-LOOP CONTROL 

Imagine driving an automobile with the windshield blacked out and with 
no “feel” in the steering wheel. Without visual and force feedback, 
catastrophe would soon result. Control under these conditions is termed 
“open-loop,” and though it would seem a disastrous approach to teleoperator 
control it is employed in special circumstances. 

One such circumstance occurs whenever the control of a teleoperator is 
relinquished by the human operator to a preprogrammed set of instruc- 
tions — say, a preprogrammed subroutine in an on-board computer that 
automatically stows a manipulator on a submersible. Open-loop subroutines 
are essential in supervisory control; in fact, the use of computers to relieve 
the operator in routine situations and provide special nonanthropomorphic 
skills is so important that we devote the next section to this subject. 

Meanwhile, teleoperators that are normally operated in a closed-loop 
mode may revert to open-loop control under the following conditions: 

1 . If feedback is temporarily cut off, based on cues acquired before the 
displays were blacked out, an operator can usually make several 
movements safely. The feedback-deprived automobile driver mentioned 
above can, for example, pull safely off the road if he knows where he 
was before the blackout and if the traffic is light. 

2. If feedback information suddenly becomes unintelligible due to noise 
or becomes too complex for the operator to cope with, the operator 
might well proceed open-loop fashion to some safe holding position. 

3. If there is significant time delay and the operator cannot discern the 
consequences of his actions for several seconds, he may adopt a 
move-and-wait strategy in which each short open-loop move is prefaced 
by an analysis of the consequences of his last move. The operation of 
the Surveyor lunar surface sampler employed this philosophy. (See 
later section in this chapter on time delay.) 

4. If a tedious repetitive task is anticipated, one cycle of the operation 
can be carried out once under closed-loop conditions, with all control 
information being recorded, and thereafter accomplished by super- 
visory control without the operator in the loop. 
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PREPROGRAMMED CONTROL 

In preprogrammed control, the operator turns control of the teleoperator 
over to a machine, one with a memory that contains instructions for carrying 
out a given order. The instructions may be stored in a computer’s memory or 
engraved in analog form on a grooved rotating disk or cam, like the famous 
Jaquet-Droz automatons in the late 1700s. The operator may transfer control 
by simply pressing a button, typewriter keys, or by reading a deck of 
punched cards into a computer. Or, in principle,, the machine portion of the 
teleoperator may intentionally bypass the operator in an emergency and 
switch in a preprogrammed subroutine. A common feature of preprogrammed 
control is the absence of any feedback to an operator that would permit any 
modification of the action — the “manipulator stow” subroutine, for ex- 
ample. Once the subroutine is in action, it is played out. In other cases, the 
human operator can inhibit action and correct errors. 

It is often desirable to initiate a subroutine which requires internal 
feedback of some sort (unseen by the operator) to carry out an instruction. 
An operator may in fact cut himself out of the loop and switch in a variety of 
supervisory subroutines, including: (1) the type of open-loop preprogrammed 
subroutine just described; (2) an automatically controlled closed-loop 
subroutine that utilizes feedback signals to reduce the task error, for example, 
the automatic movement of the teleoperator arms into configuration A; or 
(3) an adaptive or artificially intelligent subroutine that makes its own 
decisions on how to best carry out an operator’s directive, perhaps by 
transferring object X to point B around an obstacle. Closed-loop subroutines 
(2) and (3) of course require feedback, whereas open-loop subroutine (1) 
moves ahead oblivious to feedback. In effect, we have established the matrix 
of operator-machine control relationships illustrated in Fig. 12. 

Is there a formal theory of open-loop teleoperator control? There is little 
to report here. Naturally, a strategy is important in an open-loop move-and- 
wait situation like that encountered in operating the Surveyor surface sampler 
(Fig. 13). If one wishes to dig a trench on the Moon, one does not at first 
take big bites from an unknown medium that might damage the sampler 
itself. Instead, one devises a strategy composed of moves such as: Extending a 
sampler arm in increments of a half-inch at a time, waiting between moves to 
see the results.* In other words, adopt a “move gingerly” strategy. 

Beginning with Ernst’s work in 1961 s several researchers, notably at 
M.I.T. and Case Western Reserve, have interposed a digital computer between 
the human operator and the manipulator. 6,7 The software and hardware 
employed in these NASA-supported experiments will be described in 
Chapter 5. A typical open-loop computer instruction during a stow sub- 
routine or reactor core disassembly might be: Move joint C 5° clockwise. The 


* Preprogrammed tapes controUed some Surveyor sampler arm motions. 
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TYPE OF CONTROL 


OPEN-LOOP CLOSED-LOOP 


OPERATOR 
IN LOOP 

MOVE-AND-WAIT 

STRATEGY 

NORMAL TELEOPERATOR 
OPERATION 

OPERATOR 

PREPROGRAMMED. 

ADAPTIVE CONTROL; 

OUT OF LOOP 

SUPERVISORY 

ARTIFICIAL INTELLIGENCE; 


CONTROL 

AUTOMATIC CONTROL; 



SUPERVISORY CONTROL 


Figure 12 Matrix of various teleoperator control situations. To qualify as a 
teleoperator , the machine should operate with the operator out of the loop 
only in special situations where the human operator cannot cope with the 
task or where he wishes to relieve the task burden. 



Figure 13 The Surveyor surface sampler . (Courtesy of D. Le Croissette , Jet 
Propulsion Laboratory.) 

computer merely acts as a switch in this case, turning on the motor driving 
joint C for the requisite number of revolutions. In open-loop control there is 
no feedback to assure the computer that joint C really rotated 5°, although a 
limit switch would probably be installed to indicate completion of the task. 
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CLOSED-LOOP CONTROL 

Sophisticated control systems depend upon feedback; teleoperator 
controls are no exception. Teleoperators are normally operated with man in 
the loop and with visual feedback. Even many of the supervisory subroutines 
that relieve man of participation in control depend upon internal feedback 
signals to carry out their instructions (Fig. 12). 

A large body of theory has grown up around the concept of feedback 
control. 8,9 Our objective here is to summarize some of the conventions and 
the general teleoperator approach. 

The essence of feedback control is, of course, feeding some of the output 
back into the input to modify it. One tries to reduce the error with feedback, 
but sometimes this tactic is not successful and instability occurs. Some 
important control conventions are illustrated in Figs. 14 through 16. The first 
of the “block diagrams,” Fig. 14, illustrates how an input, R, is affected by a 



Figure 14 A simple open-loop control situation . 
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Figure 1 5 Two control components in series. 
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Figure 16 A fundamental equality in control circuit theory. 
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control system element symbolized by the block and is algebraically 
represented by the transfer function, G. The G symbolizes “something done” 
to the input signal. The output, C, is given by C = GR. The block diagram of 
Fig. 14 is completely equivalent to the equation C = GR. The control element 
thus represented is obviously linear. If two control elements are in series 
(Fig. 15), the overall equation is C = Gx G 2 R. The Gs are often called forward 
transfer functions and may include the human transfer function. 

When feedback exists, control theory convention calls for the addition of 
the subtractor symbol, the circle in Fig. 16. Here, the input R and the 
feedback signal, HC, are subtracted: E = R — HC; where H is the feedback 
transfer function and E is the actuating signal. Because C = EG, we can also 
write: 



This equation and Fig. 16 represent closed-loop control with negative 
(degenerative) feedback. Feedback can be positive as well as negative. Also, 
its frequency, phase, and other characteristics can be modified to achieve the 
goals of the control system designer. (For a grounding in control system 
theory, see References 8 and 9.) 

For all the simplicity of Fig. 16 and the equivalent equation, they are 
really a facade for more complex equations describing the dynamics of the 
control system as measured in terms of its input and output voltages, 
displacements, or whatever the physical parameters may be. To transform the 
usually complex equation expressed in physical parameters into the G, H, R, 
C, E representation, one utilizes the well-known Laplace transform. Again, 
the reader should consult the many textbooks on control theory. 

Conventional feedback control theory is applicable in principle to 
teleoperators with man in loop and when the teleoperator is controlled by 
automatic control subroutines that depend upon feedback, almost all extant 
tele operators fall into these two categories. We specify “conventional” 
control theory because later in this chapter we will describe some new 
theoretical developments oriented specifically toward teleoperators. 


ADAPTIVE CONTROL AND ARTIFICIAL INTELLIGENCE 

The word “adaptive” is employed fairly loosely in the control literature. 
Generally, an adaptive control system is one which adjusts to meet changing 
circumstances. In this sense, any feedback control system is really adaptive. 
In this book, however, we narrow the meaning to include only control 
systems that can cope with changing external circumstances beyond the 
capacity of simple feedback control. Two examples: avoiding an obstacle and 
finding the quickest way to take a manipulator from configuration state A to 
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state B. In other words, judgment and decision-making are involved in 
adaptive control; something beyond the ken of a “deterministic” feedback 
control system such as a thermostatic temperature regulator. The distinction, 
however, is rather fuzzy. 

Even fuzzier is the distinction between adaptive control and artificial 
intelligence. An artificially intelligent machine would not only be adaptive 
but would also have the ability to learn from past mistakes and be able to 
devise strategies of a general nature to reach goals set by itself — or perhaps 
goals set by man if the machine still depends on him at this stage of 
development. 

A teleoperator, being a man-machine system, is always adaptive and 
intelligent when man is in the loop because man has defined these 
characteristics from his analysis of himself. But when operating in a 
subroutine, we often look to the machine portion of the teleoperator to do a 
little thinking for itself. 

Before confining the discussion to machines exclusively, we would like to 
mention an interesting body of theory developing around the adaptive 
behavior of the human controller . 1 °' 12 The approach is much like that used 
in “tracking experiments”, wherein an operator tries manually to follow a 
target or reduce an error signal.* In adaptive tracking experiments, however, 
instead of merely shifting the target with time, the entire dynamic system 
may be altered and the operator has to revise his strategy in midcourse based 
upon his observation of the altered system. Eventually, this approach may 
prove useful in teleoperator control because we obviously wish to describe 
accurately how the human half of the teleoperator adapts to changing ground 
rules. 

To illustrate how feedback theory also applies to subroutines, we describe 
how the computer-controlled manipulator at Case Western Reserve assures 
that it has correctly carried out an instruction . 7 If the subroutine requires 
that the manipulator move to a specified configuration (or “state”), the 
computer compares the current configuration of the manipulator, axis by 
axis, with the desired configuration. The differences in axis positions are 
converted into analog voltages. These voltages — really error signals — drive 
the axis motors until the errors disappear. The resultant configuration should 
be the desired one since all errors have been nulled. The feedback in this 
example consists of the voltages (from axis potentiometers) representing the 
manipulator configuration as a function of time. It is classic feedback control. 
The operator, though, is not in the loop during this operation. 

The Case computer-controlled manipulator also exhibits a kind of 
adaptive behavior in its ability to avoid obstacles in its path. If the computer 


*The so-called “tracking experiments” are the foundation of manual control theory. 
In these, a subject tries to follow a target or reduce an error. See later discussion under 
Manual Control and Tracking Theory. 
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memory knows the location and configuration of the obstacle situated 
between the initial and final manipulator configuration, it will first check to 
see if other terminal arm-hand configurations can place the hand in the right 
position. If so, the obstacle may be avoided by proceeding to one of these 
directly. The computer checks to see. If obstacle avoidance is still impossible, 
the computer will explore several paths leading around the obstacle, select the 
one requiring the least transit time, and set the manipulator in motion along 
this path (Fig. 17). Clearly, a judgment and a decision have been made. 




Figure 17 Some possible obstacle evasion paths tested in the Case Western 
Reserve computer-controlled manipulator experiments. 7 


Similar obstacle-avoidance studies are being pursued under NASA 
contract by Sheridan’s group at M.I.T. using sets of heuristics arranged 
according to a priority criterion. 1 3 One heuristic approach might be to try a 
series of straight line motions tangent to the obstacle’s peripheries. 

Most of the walking machines we see today are preprogrammed and 
open-loop. They tread away blindly, regardless of the terrain. R. J. Hoch and 
his associates at Battell e-North west Laboratories have conceived of a method 
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that may make walking machines adaptable to varying terrain . 14 The germ of 
the Battelle idea lies in the short-term memory of a small computer and the 
quantizing of the control system. Control of the walking machine by pistons 
is accomplished by a series of discrete pulses, N pulses per second to each 
piston. Initially, the control pulses would be those that would carry the 
vehicle over ideal, non-varying terrain at the gait and speed set by the 
operator. In this mode, the operation would be preprogrammed; but as the 
terrain departs from ideality, the piston backpressures (the discrete feedback 
pulses) would also depart from those expected from an ideal terrain. The 
differences between the ideal and the real signals would be stored in the 
control computer memory and used to modify subsequent control pulses. 
The older the differences the less their weights in determining the next cycle 
of control pulses. The Battelle scheme would also employ sensors that feed 
back data on vehicle stability that may bypass the normal controls in favor of 
some emergency subroutine — say, one that prevents the vehicle from 
overturning. The use of past deviations from ideality in determining future 
action is a form of learning. We humans are adaptive walking machines, 
except that we can usually see the terrain ahead and add this knowledge to 
that from past experience. Note that the Battelle walking machine would not 
have the human operator in the loop while controlled by the computer; it 
would operate under supervisory control during these periods. 

A different approach to adaptive control has been proposed for artificial 
arms by Lyman’s group at the University of California at Los Angeles . 15 
Here, the strategy is to increase or decrease the control system gain as the 
average operator error decreases or increases, respectively. The error is 
measured in terms of overshoot and undershoot of a target. In this case, the 
machine adapts itself to the operator’s skill. The operator “trains” the 
machine here, whereas the Battelle walking machine was trained by the task 
itself. 

Since the attainment of intelligence by a machine is a matter of debatable 
definition, many will say that an adaptive walking machine is definitely not 
intelligent. Yet, the Battelle approach permits the redirection of action based 
on learned facts. In the usual case, a teleoperator would not be an artificially 
intelligent robot because man is in the loop adding his brain. One can 
conceive of situations, though, wherein man divorces himself more and more 
from detailed control, giving more and more executive orders, such as “pick 
up object A and move it to point B.” Considerable work of this type is 
underway. At MJ.T.’s Project MAC, Minsky is building an autonomous 
robot ; 16 so is Rosen at Stanford Research Institute . 17 One can consult 
Feigenbaum and Feldman 18 for discussions of specific machines like the 
General Problem Solver and the Logic Theory Machine. Almost all of today’s 
teleoperators require the full real-time involvement of man; computer control 
of teleoperators with nearly autonomous machine partners has been 
demonstrated but is still far in the future. 
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STABILITY 

Stability 7 exists in a control system when the transients created by a 
disturbance eventually die out. If the transients do not completely die out, 
but remain bounded, limited stability exists. Instability plagues only 
closed-loop control systems, because only when the system output can 
reinforce the input can divergent behavior occur. Most texts on control 
system design devote considerable space to stability criteria and the Nyquist 
diagrams that predict the stability of a control system. 8 The reader should 
consult these texts for details. 

Teleoperators, being mainly closed-loop systems that incorporate man, 
naturally face stability problems. The intimacy of the man-machine interface 
poses a special problem in teleoperator design. A human operator may make 
mistakes, get tired, become confused, and otherwise contribute to instability. 
The time delay of feedback data is a case in point because too great a time lag 
disconcerts the operator and may stimulate divergent transients. 


THE TIME DELAY PROBLEM 

Many people have experienced the disconcerting effects of delayed audio 
feedback, particularly in public address systems. Delayed visual and force 
feedback can compromise teleoperator control in a similar fashion. In this 
chapter, we define the^ problem and look at some “preview control” models; 
Chapter 6 covers the predictor displays that have been designed to help solve 
the time delay problem. 

NASA has been concerned with transmission delays resulting from the 
finite speed of radio signals over the great distances in outer space. Between 
Earth and Moon, the round-trip signal time is roughly 2.6 seconds — enough 
to disconcert an Earth-based operator of a lunar machine (Fig. 18). 

Besides the signal propagation time delay, feedback information also 
encounters electrical circuit and mechanical device delays. The operator in 
the control loop also slows signals down. Wargo has summarized human 
delays for one-choice situations as follows: 1 9 


Receptor delays 

1—38 milliseconds 

Afferent transmission delays 

2-100 

Central process delays 

70-100 

Efferent transmission delays 
Muscle latency and activation 

o 

to 

o 

delays 

30-70 

Total delay 

113—328 milliseconds 
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TOTAL DELAY - SECONDS 

Figure 18 Normalized task time vs. total time delay. A critical region, 
around 0.25 sec ; where operator confusion is possible , occurs in space 
missions at very high orbits. Move-and-wait operator strategy would be a 
successful but slow strategy for work on the Moon and planets. 

These figures are for alert, pre-warned subjects. Reaction times lengthen when 
the unexpected occurs and when the choice is a complicated one. Human 
delays and equipment delays are usually much smaller than those NASA 
anticipates from propagation delays in space exploration. 

The portion of the overall time delay that disconcerts the operator is that 
part that prevents him from seeing the immediate consequences of his 
actions. Ferrell and others have summarized past work in the field of delayed 
sensory feedback. 20,2 1,22 The early studies involved tracking experiments 
of various kinds with delayed visual feedback. All of the studies concurred 
that time delay was deleterious to performance. In connection with its 
projected remotely controlled lunar vehicles, NASA has sponsored work at 
Stanford University 23 which indicated that driving performance worsened 
with increasing time delay. The situation deteriorated faster as vehicle speed 
increased, as the vehicle course became more complex, and as the television 
field of view narrowed. Similar effects have been noted for delays in auditory 
feedback. Recent theoretical work by W. H. Thompson at ANL indicates that 
force feedback, too, is of diminishing utility as time delay increases (see 
Bibliography). 
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Some pioneering work with the effects of time delay on whole-arm 
motions (many degrees of freedom) have been carried out by Karl U. Smith, 
at the University of Wisconsin. Time delay can apparently be understood in 
terms of feedback theory and the perturbation of the operator’s internal 
synchronization. 

Also applicable to teleoperator control were Ferrell’s 1963— 1964 
experiments with a two-dimensional manipulator with variable time delay in 
the visual feedback. 20 The manipulator operators in this case were able to 
pace their activities and work out strategies that suited them best. The 
tracking and vehicular experiments, in contrast, force the operator to 
synchronize his activities with the input signal* Ferrell found that task 
completion time increased with both time delay and task difficulty (Fig. 19). 



Figure 19 Task completion times as a function of task difficulty from 
FerrelVs experiments . 2 0 


Some important conclusions from Ferrell’s work were: 

1. Both operators independently adapted a move-and-wait strategy as the 
best and least confusing solution to the unnatural time-delay situation. 


2. There were no unstable or oscillatory movements. This fact was 
attributed to the adoption of the open-loop move-and-wait strategy. 


*This is one reason why tracking experiments in the manual control field have 
limited application to tele operator control; an application where the operator moves at 
his own speed. 
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3. The operators found the work tiring and difficult, but not emotionally 
upsetting as other operators have reported for forced-pace time-delay 
tracking experiments. Trials where these operators were asked to use a 
move-slowly strategy did, however, disconcert them. 

In summing up, time delay (and task difficulty) can be overcome by taking 
additional time — mostly waiting time between successive operator moves and 
the returned feedback. 

While it is not yet apparent just how tracking theory and experiment 
apply to teleoperator control, numerous tracking experiments involving time 
delay have been carried out and some “modified” human transfer functions 
have been generated. 24,25 Note that there has been no attempt to evolve a 
mathematical model for manipulatory task performance involving time delay. 

Suppose that the extra time required by the move-and-wait strategy is 
unacceptable; what can be done? In Chapter 6, we will describe some 
predictor displays that aid the operator working under a time-delay handicap 
by giving him feedback extrapolated into the future on the basis of known 
physical laws. In the present chapter on theory, we look at the so-called 
“preview models,” which are really surmises on how an operator extrapolates 
his machine, the environment, and himself into the future. 

The basic ideas behind preview models were published by Ziebolz and 
Paynter in 1953. 26 Kelley adopted these ideas in his Predictor Instrument in 
I960. 27 Particularly pertinent to the teleoperator field are the preview 
models of Sheridan which incorporate prediction and planning aspects. 28 
Sheridan’s three models may be termed: (1) the “extended convolution” 
preview model; (2) the “dynamic programming,” optimal trajectory, preview 
model; and (3) the Ziebolz controller preview model. The Ziebolz controller, 
which is a key ingredient in many preview models, is essentially a fast-time 
predictive model (or analog) of the system under control. To date, preview 
models have been developed around tracking experiments, where theorists 
have some confidence that they have valid mathematical representation of the 
control process. As mentioned before, the application of these models to 
teleoperator tasks is questionable. 

One might summarize the time-delay situation by noting that: (1) the 
problem must be solved if teleoperators are to be effective over long 
distances; (2) current theory has so far offered little help; but (3) operators 
facing the problem solve it naturally by moving and waiting. 


MANUAL CONTROL AND TRACKING THEORY 

Earlier in this chapter, we have occasionally mentioned tracking 
theory — perhaps a little too disparagingly. Nevertheless, modem manual 
control theory is largely built upon a foundation of tracking experiments. 
These quantitative experiments form the only real basis for evolving and 
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testing hypotheses in manual control. And manual control theory is the only 
kind of control theory we have that includes in the loop the human operator 
with all his idiosyncracies. 

Three main types of tracking are recognized: 

1 . Pursuit tracking, wherein the operator sees both the moving target and 
his own corrective responses (Fig. 20). A common analogy is a duck 
hunter using a gun with an open sight. In laboratory practice, the 
operator tries to follow a moving target, say, a moving spot, using a 
joystick or some other control. 

2. Compensatory tracking, in which the operator sees only the differences 
between the moving target and his response; i.e., the error (Fig. 21). In 
this type of tracking, the operator attempts to null the difference 
signal. 

3. Precognitive tracking, which exists when the operator has complete 
information about the target’s future — as in shooting at a duck in a 
shooting gallery. In the true sense of the word, this is not really 
tracking. 

Which of these kinds of tracking have application to teleoperator control? 
Pursuit tracking applies if the teleoperator is trying to pick up or perhaps hit 
a moving object, a very rare situation in present-day teleoperator practice. In 
manipulation, the targets are generally stationary; so is the environment. In 
picking up an object, the operator first directs the manipulator hand to the 
general area of the target in a gross movement; then, in a series of fine 
adjustments, the hand is accurately positioned and the jaws closed. The same 
kind of coarse-fine “tracking” occurs when the target is moved from position 
A to position B. But which of the three main varieties of tracking describe the 
situation best? Obviously, precognitive tracking is closest, but it is not bona 
fide tracking at all. No formal theory exists for precognitive tracking. 



Figure 20 In pursuit tracking , the subject tries to align the solid vertical 
trace with the target circle . 
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Figure 21 In compensatory tracking , the subject tries to null the error , E. 


There are, however, elements of pursuit tracking that may be applicable. 
For example, the First gross movement of the manipulator hand to the region 
of the target is akin to getting the duck in the gun sights, and the fine 
motions prior to grasping the target really involve nulling out the position 
errors the operator sees visually. In sum, there is no single type of tracking 
that seems to cover teleoperator action. Further, there is no theory at all that 
really grasps the planning and strategic thinking of the person controlling a 
teleoperator. 

These things being so, why bother to discuss tracking theory at all? The 
answer must be that tracking theory gives us the only quantitative insight into 
the behavior of humans in control loops, despite its acknowledged drawbacks. 
Any comprehensive theory of teleoperator control (including the operator) 
must build on (or alongside) manual control theory. 


THE HUMAN TRANSFER FUNCTION 

Tracking theory got its start in England and America during World War II 
in connection with gun pointing or “fire control.” Elkind has reviewed the 
development of tracking models 29 and notes that despite some twenty years 
of research the “...relative lack of progress is not so much a result of lack of 
interest and effort as it is a consequence of the complexity of the human as a 
controller and of the interaction between him and the rest of the control 
system.” During World War II, Tustin introduced the idea of representing the 
dynamic response of a human operator by a linear transfer function. 30 Since 
the War, most effort has gone into a search for a human transfer function in 
tracking tasks. To review this early work, the reader should consult such 
review articles as those by McRuer and Krendel 3 1 and Sheridan. 32 

The aim of manual control theory — or teleoperator control theory for 
that matter — is the description of the human operator in terms of a human 
transfer function that will put him on a par with other predictable control 
circuit elements. 




CONTROL THEORY 


37 


In attempting to describe the human as a servo element, a tracking 
experiment is set up that places the subject in a control loop, provides a 
variable input or stimulus, and measures the human response. Classically, this 
is done with a continuous input signal that varies over a wide frequency range 
or bandwidth. The experimental response in terms of bandwidth, phase 
change, and so on, leads to the human transfer function. The literature in this 
field is so voluminous (see References 33 to 36 for review articles) that we 
can only touch on a few high points. 

The hope of most theoreticians and experimentalists is for linearity. 
Tustin’s early work in the late 1940s (and later studies) discovered a linear 
term in the human transfer function approximated by: 

H(s) - Ke 78 (1 + T l s) 

where H(s) = the human transfer function as a function of the Laplace 
variable, s 

K = the gain coefficient, equal to about 22 

r = a time delay term, equal to about 0.3 second for Tustin’s work 
T L = a lead term, equal to about 2.3 

Tustin also found a remanent term which was not linearly related to the 
input. The schematic representation of this situation is shown in Fig. 22. The 


i 1 



Figure 22 Diagram of the widely accepted quasilinear model of the control 
circuit containing a human operator . 

situation was further complicated by higher frequencies in the output than 
had been in the input, obviously these were operator-induced. Delay times 
were often much greater than 0.3 second. Furthermore, output was often 
disproportionate to the input, sometimes even of different sign! 
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Many other studies followed Tustin’s. 31,37 A simple, generally accepted 
form of the linear portion of the transfer function is now taken to be:* 


H(s) = 


Ke^ s (1 + T L s) 
(1+T n s)(1+TjS) 


where T n “ a neuromuscular lag time constant 
Tj = a compensatory lag time constant 

When expressed in terms of output ( 6 0 ) and input (0^) rather than the 
Laplace transform, the relationship is: 


0o(t) + (T N + T I )^^+T N T I 


d 2 6o(t) 

dt 2 


= K[0j(t - t) + T l 

In modem compensatory tracking experiments r averages about 
0.15 ± 0.03 second. A good operator tracks poorly when the input exceeds 
one cycle per second (see Fig. 23). For discussions of experimentally 
determined values of the other parameters, see Kelley’s book. 2 7 

One question that inevitably arises is whether the human transfer 
function applies to situations involving more than one loop or degree of 
freedom. This is certainly the case in any practical teleoperator. Several 
studies have been performed for multiloop situations. 38,39 These generally 
show the classical human transfer function is not as well confirmed as it is in 
single-loop situations. 

The models of the human operator discussed above are usually charac- 
terized as “time-invariant and quasilinear.” In other words, the operator is 
assumed to be a static, nearly linear circuit component; and of course he is 
neither. Sheridan, 40 McRuer 37 and others have tried to take into account the 
tendency of the operator to change his transfer function as the situation 
demands. Man “adapts.” He also gets tired with time. Obviously, the human 
operator is far from time-invariant. So far, these efforts have not produced 
any widely useful time-dependent models. 

Most tracking experiments employ visual feedback. Tele operators, how- 
ever, frequently supply the operator with force feedback. What little is 
known about the human transfer function in the presence of force or audio 
feedback indicates that the general mathematical form obtained in visual 
experiments remains satisfactory. 


* There are many other models— some considerably more complex. 
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Figure 23 Amplitude ratio and phase angle vs. input signal frequency of a 
typical human operator describing function. 31 


Early studies tacitly assumed that the human controller is a continuous 
circuit element. Evidence exists, however, that human perception, perhaps 
cognition and other human functions, are discontinuous. These observations 
have led to the so-called “sample d-data” model of the human control- 
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ler. 41 ’ 42 The applicability and utility of the sampled-data approach are 
controversial subjects at present. Some experiments do not support the 
model, some do. Fogel’s recent NASA-sponsored work should be consulted 
for the “pro” side of the argument. 42 

The critical question after all this exposition is whether the models of the 
human controller can help us design better teleoperators. The human transfer 
function approach has only very limited applicability to manipulation, 
regardless of its demonstrated success in describing single-loop tracking 
experiments. In reality, it does not try to describe manipulation at all. The 
tracking experiments that determine the human transfer function and the 
constants in its mathematical representation do give us some insight into the 
servomechanism properties of the human operator. Human reaction time and 
bandwidth, for example, come out of tracking studies, although they depend 
upon the situation and must be used with caution. At best they can be 
considered “calibration functions.” Regardless of the drawbacks of human 
modeling (Table 4), a start has to be made somewhere. 


Table 4 — Comparisons Between Human Transfer 
Functions and a Real Human Operator* 


Human Transfer Function 

Human Operator 

Input has same number of dimensions 

Typically input has more dimensions 

as output 

than output 

One display or feedback channel 

Multiple feedback channels 

Assumes impoverished display 

Sophisticated multidimensional 

format 

displays 

Does not include explicit 

Operation vitally affected by 

representation of task or 

understanding of task and environ- 

environment 

ment 

Restricted to present error, fixed 

Response based on remembered past 

weightings of past, and 

and predicted future 

derivatives 


Cannot remember; can only sum- 

Can remember, modify response on 

marize signals by integration 

basis of past experience 

Cannot predict input or output; 

Can predict and adjust response 

response is an arbitrary 

to minimize future error. 

weighting of error, lead. 


and lag terms. 



*Adapted from Kelley . 27 


SOME APPROACHES TO TELEOPERATOR 
CONTROL THEORY 


Granted the weaknesses and general inapplicability of classical manual 
control theory to teleoperators, what has been done in the way of 
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formulating a useful description of teleoperator control processes? Not a 
great deal! This should not be too surprising, because the human functions of 
planning and strategy setting are still being argued and have yet to be 
embraced by mathematics. 

Seidenstein and Berbert 43 have examined the extant literature in those 
areas which they believe comprise the most important “extra” dimensions of 
teleoperator control: 

1. Judging the best path for approaching the target. 

2. Approaching the target and minimizing undershoot and overshoot. 

3. Orientation of hand for manipulatory task. 

4. Final adjustment of arm and hand. 

In 1966, Seidenstein and Berbert found essentially no important 
literature that would give a foundation upon which to build a comprehensive 
theory of tele operators. However, their literature review did not encompass 
K. U. Smith’s work and much of the psychomotor theory of perceptual and 
motor organization (see Bibliography). Some of these theories may ultimately 
prove extremely useful in teleoperator theory. 

Significant direct attacks on the tele operator problem have been made by 
Sheridan’s group at M.I.T. and Lyman’s group at UCLA. 44,45 Both groups 
bypass the human operator as a planner and strategy formulator. Although 
man still retains “executive” control of the operation in their approaches, 
their theories concern only the machine part of the teleoperator in 
supervisory control situations. 

Sheridan compares the different kinds of manual control diagrammati- 
cally, as illustrated in Fig. 24. It is the last schematic, representing supervisory 
control, which focusses the different theoretical approaches. With man out of 
the loop and with the tele operator configuration and the environment 
(including the targets) described in “state space” one can apply some 
powerful algorithms like those that have been developed for playing chess 
with computers, finding the quickest route between points, etc. In other 
words, there is no theory that describes how a human solves a manipulatory 
problem, but we have developed theoretical approaches which can guide 
machines while they perform the same problem! If a human is truly a 
machine, as Wooldridge contends 46 perhaps we may eventually be able to 
apply some of these theories to ourselves, and thus improve our own 
capabilities. 

One of Sheridan’s students, D. E. Whitney, has completed some 
pioneering work in the field of supervisory manipulation in state space. 47 He 
lists the following attributes of a good computer-controlled manipulator, 
which sound remarkably like the qualities a human manipulator operator 
must have: 
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Figure 24 Comparison of vehicle control and several varieties of manipulator 
systems, (a) Vehicle control; (b) direct manipulation; (c) remote manipula- 
tion; (d) supervisory manipulation , 44 

l.It employs a symbolic representation or model of the task site. All 
objects, obstacles, fixed support surfaces and effectors (jaws, tools, 
etc.) are represented in their proper spatial relationships. 

2. It can identify goals in this model. A goal may be thought of as a 
particular configuration of the objects, obstacles and effectors which 
the operator wishes to attain. 

3. It understands how the effectors can alter the task site as well as how 
these alterations are represented in the model. 

4. It can receive commands which specify goals to be achieved and 
constraints to be obeyed. Then, using items 1, 2, and 3, it can translate 
the command into an expanded equivalent. (“Expanded” means that 
strings of manipulator primitive commands have been substituted for 
the human primitive command; “equivalent” means that these manipu- 
lator primitive commands, when carried out, can be expected to 
accomplish the previously stated goal.) In other words, the system can 
make a plan for carrying out the task. 
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5. It can execute this plan, judging its progress against the plan’s 
expectations, keeping track of its progress by updating the model, and 
asking for help if trouble develops or things do not go according to the 
plan. 

Suppose the manipulatory task is to move an object from one point to 
another on a table, avoiding an obstacle on the v/ay (Fig. 25). In state space 
coordinates this means moving from 



'X 0 ‘ 


[*fl 

x(t 0 ) = 

yo 

0 

to x(t f ) = 

yf 

0 


.0 . 


.0 . 


between time t 0 and time tf* The zeroes in the state vectors represent 
velocity components. The values of x and y, however, must not assume values 
off the table or too near the obstacle. The best trajectory is found by testing 
each trajectory between the two points which satisfy the constraints and 
comparing one against the other using some time, distance, or cost criterion. 
Problems such as this are common in engineering and are solved by the 
methods of “optimal control,” including the calculus of variations, dynamic 
programming, etc. 48 


y 

t 



Figure 25 Diagram showing possible paths around an obstacle in Whitney's 
state-space approach 

A more complete recounting of a typical manipulatory task — carrying 
object A to location X — is shown in Fig. 26. Note that the diagram is 
basically a computer program and really recapitulates a human operator’s 


*State space also includes hand position and orientation as well as all features of the 
environment 
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Figure 26 A possible program for instructing a computer to carry object A 
to location X 47 


unverbalized considerations. Optimal control enters the picture at the point 
where the motion plan is being formulated. 

“Inhibitory control,” proposed by Lyman and Freedy, 45 assumes that 
some paths between initial and final states (teleoperator plus environment) 
are more likely than others. Our eating motions, for example, are rather 
stilted, and this is true of most routine manipulatory functions, especially 











CONTROL THEORY 


45 


those by wearers of artificial limbs. In inhibitory control, an adaptive 
controller — probably a computer — would drive the tele operator between 
the initial and final states (initially selected by the human controller) along 
historically favored paths. The human operator monitors the motion and 
inhibits it where it is in error, due perhaps to a new obstacle placed in the 
environment. The adaptive controller adds this new information to its 
running account of favored tele operator motions; while the human operator 
still monitors the activity, he is relieved of the burden of planning and 
detailed execution of the task. This approach differs from that of the M.I.T. 
group in that human judgment guides the choice of path rather than some 
optimal control scheme based on minimum time or some other constraint. 

APPLICATION OF CONTROL THEORY TO 
UNILATERAL TELEOPERATORS 

Unilateral teleoperators (sometimes called “rectilinear” in error) are 
controlled by 

1 . Switches* or potentiometers which actuate motors driving the various 
degrees of freedom. Feedback in this instance is visual as the operator 
corrects errors in position and orientation. If switches are used, this 
control technique is called “rate” or “velocity” control; potentiom- 
eters permit variable motor speeds or “proportional rate” control. 

2. Replica or prosthetic-type controls that are analogs of the actuator 
subsystems. The servomotors driving the various degrees of freedom are 
actuated by an error signal which is proportional to the difference 
between the desired configuration specified by the controls and the 
actual configuration of the teleoperator. This is termed “position 
control.” 

In any real teleoperator control system, the differential equations 
describing the motion of the arms, hands, and legs are complicated by the 
fact that these appendages have mass (which may lead to overshooting the 
target), have friction in the joints, and may move in an appreciably viscous 
medium, such as seawater. Such considerations are part and parcel of the 
design of most control systems, such as those of radar antennas and guns. 
Thus, the system may be damped to reduce overshooting or “hunting,” yet 
excessive damping will cause undesirably sluggish response. A compromise 
must be found. The reader should consult texts on control system design. 8 
Only a few specialized reports have been published on the application of 
control theory to specific tele operators. 1 5 ,48 

The just mentioned theory, though well-developed, excludes the most 
important control-loop component: the human operator. Since we have no 


*See Chapter 5 for descriptions of typical control hardware. 
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practical, analytical way of incorporating the human operator into the 
teleoperator control equations, conventional control theory remains only a 
guide. All is not lost, though; because when the mathematics become 
intractible, insight can frequently be gained by simulating control systems 
electromechanically. With a human operator plus a reasonable analog of the 
electrical and mechanical components, different control schemes can be 
compared, stability regimes can be investigated, and even the analytically 
elusive properties of the operator can be studied. 

Unfortunately, little basic simulator work has been completed. The most 
significant studies are those by Ritchie, Inc., under Air Force contracts, 43 ,s 0 
and at General Electric, under DOD sponsorship. 5 1 The Ritchie simulators 
employed three and four degrees of freedom and incorporated manipulator 
arm mass, damping factors, and motor characteristics (Fig. 27). The following 
conclusions are taken from the Ritchie studies: 
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Figure 27 Sketch of the geometry of one of the Ritchie manipulator 
simulations , 43 
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1. Proportional rate control is better than fixed rate control and position 
control in terms of task time required and task efficiency. 

2. The use of high speeds in approaching the target resulted in longer fine 
adjustment times. 

3. Small targets require higher travel times than large targets with fixed 
rate control, but the opposite was found with position control — an 
“illogical” result. 

4. There seemed to be an optimum rate of motion (4 inches/second) for 
the conditions of the experiment. Overall performance decreased above 
and below this value. 

These kinds of conclusions, one should note, have not yet been extracted 
from current theory. 

At General Electric, R. S. Mosher’s group has come to grips with the 
practical design of the powered exoskeleton, Hardiman I. Har diman’s leg 
joints are unilateral, while the arm and hand joints are bilateral. To 
complicate the situation some of the joints are in series. In this section, we 
briefly describe some of the theoretical studies and simulator work done for 
the three-joint unilateral case. 

Figure 28 illustrates the signal flow diagram for the three joints. The 
dynamic cross coupling connections are significant because the dynamic 
feedback to joints 1 and 2 may be positive, creating instability. Note that this 
feedback is not through the operator or control circuits but rather is a 
consequence of the series connections of the joints; that is, a jointed, 
unilateral teleoperator arm could dynamically tear itself apart. General 
Electric’s simulation of this situation indicated that instability could indeed 
result with the high gains required for control of Hardiman I, and that this 
instability could be prevented by the addition of proportional, rate, and lag 
compensation networks. 

In the General Electric work we get an inkling of how complicated 
tele operator control theory can be even though the human operator was not 
included in these simulations at all. 


APPLICATION OF CONTROL THEORY TO 
BILATERAL TELEOPERATORS 

The first bilateral servoed teleoperators* were built by R. C. Goertz’s 
group at Argonne National Laboratory (ANL) in the early 1950s. Several key 
theoretical papers originated from this work, 1 * 2,53 which was very extensive. 
We can show only the general approach here. Following Burnett, 54 the 


*The common mechanical master-slaves are also bilateral but they are not servoed. 
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three unilateral joints in series . 
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somewhat idealized symbolic diagram for the ANL Model 2 force-reflecting 
electrical master-slave manipulator is presented in Fig. 29. The dynamic 
equations in Laplace transform notations are: 

T x -K t I = (Js + F)s 0! 

T 2 + K t I = (Js + F) s0 2 

V = Kj (i Oi -6 2 ) + K 2 s(6 1 -0 2 ) 

IR - V + K b s0! — K b s0 2 

where J = the motor and gear train inertia 
F = the mechanical viscous friction 
s = the Laplace transform variable 
K t = the torque constant 
K b = the back EMF constant 
Ti ,T 2 - externally applied torques 
diJd 2 = angular displacements 
R = resistance 
I = current 
V = voltage 

Ki,K 2 = constants defined in Fig. 29 

The first two equations are torque equations; they assume linearity and 
complete bilateral symmetry. The equations represent what is termed a 
first-order (linear) analysis. Stability is indicated, but higher order (non- 
linear) analysis could reveal instabilities leading to oscillations. 

When bilateral joints are connected in series the analysis gets even more 
complex. The arms of Hardiman I (Fig. 30) have three such joints. The signal 
flow and block diagrams are too involved to reproduce here, and the reader is 
referred to the original General Electric report. 5 1 In fact, General Electric did 
not try to analyze the three-joint bilateral model; instead the engineers 
extrapolated the results of the three-joint unilateral and single joint bilateral 
cases. The three-joint bilateral model was simulated on an analog computer. It 
was found that the compensation networks described earlier for the unilateral 
case also stabilized the bilateral model from 0—1500 pound loads. Teleopera- 
tors like Hardiman I are feasible according to the General Electric study; 
however, during the design process, provisions should be made for adjusting 
the proportional, rate, lag, and velocity feedback terms over wide ranges. 

Apparently, any practical, real-world teleoperator will defy rigorous 
analysis by virtue of its complexity, at least until better analytical techniques 
are worked out. The presence of a non-linear, time-varying human operator 
only worsens the prospects. Thus, the major conclusion of this chapter must 
be that pure analysis can only guide teleoperator design in terms of 
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Figure 29 Symbolic diagram of the force-reflecting servo used in the ANL 
Model 2 electric bilateral master-slave . 


Yi 


v 2 


Y3 



Figure 30 Sketch of the three-joint teleoperator arm simulated by General 
Electric in their Har diman studies. Each of the three joints in the actual man 
amplifier would be powered and would incorporate force feedback. 5 1 


pinpointing design weaknesses and helping the designer think out and grasp 
the interrelations among control parameters. 

The situation is not hopeless because even the three-joint bilateral case 
can be simulated. Even better is an engineering mockup of the teleoperator 
with a human at the controls. A good, general approach to teleoperator 
control design would be threefold: (1) limited analysis; (2) simulation, and 
(3) engineering mockup. 










Chapter 4 


THE MAN-MACHINE INTERFACE 


THE NATURE OF THE PROBLEM 

As computers and other machines assume more and more importance in 
our lives, the body of literature discussing the man-machine partnership and 
man-machine symbiosis grows. In the preceding chapter, it was obvious that 
the control-theory describing the total man-machine system is rudimentary at 
best. We try to describe man with the same kinds of equations we use for 
machines, but success still eludes us. In teleoperator theory, man and machine 
seem analytically irreconcilable; and to make the schism seem more complete 
few men doubt that they are superior to machines in many important ways. 
Yet, man and machine must be integrated, especially in the teleoperator 
where the partnership is closer than it is in most man-machine systems. 

Man and machine meet at two hardware interfaces in the tele operator: 
the controls and the displays (Fig. 8). Specific control and display hardware 
are covered in the next two chapters. In this chapter, the general, more 
philosophical problems of matching man and machine at these two points 
will be discussed. 

Should we match man to the machine or the machine to man? (This 
question is overworked in today’s literature.) The answer, of course, is that 
we do both to that degree needed for best teleoperator performance. 
However, because we still do not understand machines well and know 
ourselves even less well, this brave plan cannot be consummated easily — and 
then only very imperfectly. Even in our ignorance, though, we can approach 
the problem in an orderly fashion by: (1) describing the pertinent properties 
of man and machine; (2) rationally allotting tasks to one or the other; and 
(3) building sound bridges across the interfaces at the controls and displays. 


DEFINING THE HUMAN OPERATOR AND THE MACHINE 

Whenever a subject is either controversial or not amenable to precise 
description, the literature is abundant; this is the case with man as a 
controller. Fortunately, a recent and thorough survey of this field has been 
completed by Serendipity Associates under a NASA contract. 55 We lean 
heavily on this survey in this chapter, making use of those portions applicable 
to teleoperators. 
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We are trying to define the man-machine interface and just where man or 
machine should assume responsibility in a tele operator . 5 6 To this end, we list 
pertinent man and machine attributes side by side for the sake of easy 
comparison. 

Sensory Comparisons 


Man 

Senses limited to the ranges and 
characteristics specified in Tables 5 
through 7. However, these limitations 
do not affect teleoperator control 
significantly (except in underseas 
work) because properly designed 
displays can overcome most limitations. 

Man’s input channel capacities in all 
senses are limited. They can be 
saturated easily. He may need machine 
help at times. 

Resistant to jamming and noise. Man can 
often filter out the signals he wishes 
to use. 

Man can sense and recognize patterns, 
color codings, and written or printed 
characters. Targets can often be 
discerned amid noise and clutter. 

Man is usually considered to be a 
single-channel detector at any given 
instant, implying that he must switch 
his attention from one channel to the 
other. However, sight, sound and 
touch usually work together easily 
in manipulatory tasks. 

Man’s sensory capabilities are affected 
by fatigue, general health, noise, and 
other environmental factors. 

Man’s senses cannot be calibrated 
reliably in absolute terms to provide 
quantitative data. 


Machine 

Sensory ranges extend far beyond those 
of man. A machine can also sense 
X-rays and other environmental 
factors normally invisible to man. 


Machine channel capacities can be made 
as wide as desired at a price measured 
in power, weight, cost, etc. 

Generally more subject to jamming 
and noise. 

Pattern recognition possible, but 
not well-developed yet 


Machines can handle many channels 
simultaneously. 


Machines are less affected by the 
environment and wear. 

Instruments can be accurately 
calibrated and easily read. This 
may be an advantage in delicate 
manipulations. 


Sense Interpretation Comparison 


Humans often see only what they expect 
to see and can be fooled by such things 
as optical illusion. 

If a new, unexpected situation (a new 
“universe”) is encountered, man can 
cope with it better than a machine. 

An emergency or accident would fall in 
this category. 


Machines are much more literal in 
their interpretive functions. 

Generally, machines can deal only 
with the known and expected — the 
known “universe.” 



THE MAN-MACHINE INTERFACE 


53 


Man’s interpretation of data depends 
upon his previous history with them. 
Experience's usually beneficial, 
though it can prejudice an operator. 

Man’s reliability as an interpreter 
depends upon his emotional state and 
fatigue. 

Written language, color codes, and other 
symbols are readily interpreted. This 
is particularly useful in handling 
coded objects. 

Given the symptoms, a human can 
troubleshoot a malfunctioning 
teleoperator. 

The human operator can hypothesize. He 
can ideate. He can suggest alternative 
modes of action. 

Men are poor monitors of infrequent events. 

The human operator is poor at monitoring 
continuous signals and processes over 
long periods of time. 

Man is good at detecting deviations from 
normal, particularly in the presence 
of noise and other signals. 


Historical information can affect 
interpretation by machine only in 
those ways which can be implemented 
by computers; i.e., time averaging, 
etc. 

Machines are more objective, 
tireless, and unemotional 

Languages, codes, and abstract 
symbols can be interpreted only 
with difficulty. 

Machines can also do this but only to a 
limited extent. 

Machines cannot do these things well. 


Machines are much more reliable as 
monitors. 

Machines are so good at monitoring 
that some have suggested that they be 
employed to monitor men instead of 
vice versa. 

Machines are better than men at 
monitoring simple processes, but 
they are less successful when patterns 
and symbols are involved. 


Information Processing Comparison 


Relatively low-speed information processor. 
Essentially a single-channel processor 
at any instant 

Weak and inaccurate as a computer. 

Tires quickly; especially in routine, 
boring jobs. 

Man is easy to program. He does not 
require extremely precise instruc- 
tions. He is flexible. 

Information can be processed in a wide 
variety of formats. Special coding, 
punching, etc., not necessary. 

Man’s bandpass is about three radians 
per second. He can transmit 30-35 
bits/ sec. 

Man’s short-term memory is limited in 
size, accuracy and permanence. Access 
time is relatively high. 


High-speed information processor. Can 
handle many channels simultaneously. 

Tireless and fantastically accurate 
in comparison to man. Man should 
never compute if he can get a machine 
to do it. 

Programming machines is time 
consuming. Each instruction must be 
detailed and specific. 

Computers are very specific and 
limited in the forms of information 
they will accept 

A machine’s bandpass and data rate 
can be made much larger than 
man’s — at a price. A machine can 
thus potentially manipulate much 
faster than man. 

Machine memory can be almost 
unlimited. Accuracy and performance 
are high. Access time is very low. 
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Man processes information so slowly that 
he is relatively inefficient in search 
tasks, although he is good at 
recognizing and identifying targets 
once they are located. 

Man has an excellent long-term memory 
for related events. Generalized 
relevant patterns of previous 
experience can be recalled to solve 
immediate problems. 


Machines can rapidly search huge 
quantities of data for well-defined 
targets, but accuracy suffers as 
target definition is worsened. 

This property can be built into 
machines only at great expense. 


Decision-Making Comparisons 


Man can generalize and employ inductive 
processes. 

A human being does not always follow an 
optimal strategy — usually because 
he cannot perceive or examine all 
ramifications of a situation and 
cannot compute all the possible 
solutions. 

Decisions can be made despite incomplete 
information and where the rules are 
not certain. 

Human decision-making time is relatively 
high. Often man wavers between alterna- 
tives if the decision is not clear-cut 

Man is always needed to set priorities, 
establish values, set goals, risks. 

Targets of opportunity are recognized 
better by man. 

Humans can improvise superbly. 

Man learns from past experience. 

Human operators prefer tasks with high 
degrees of responsibility and 
authority. Pride and a need to prove 
“human value” are factors here. 


Machines have less capability for 
induction and generalization. 

Machines always follow built-in 
strategies, or they can compute 
optimal strategies given sufficient 
information. 


A computer usually demands complete 
information before making a decision. 

Machines are fast and specific. 


Machines must be instructed as to 
priorities, values, goals, etc. 

Machines are relatively insensi- 
tive to unspecified opportunities. 

Machines improvise poorly. 

Machines can learn, too, but are 
not proficient at it yet. 

Degrees of responsibility and 
authority are irrelevant to machine. 


Controlling Comparisons 


Cannot exert large well-controlled 
forces. (Force or pressure is man’s 
primary control mechanism.) 

Superb at manipulation, construction, 
creative work, non-routine tasks. 

Tires quickly. Easily bored by routine, 
repetitive tasks. Man is easy to 
overload. 


Machines can exert considerable 
force with speed, steadiness, and 
precision. Reaction time is much 
smaller than man’s. 

Good at routine and well-defined 
tasks; i.e., those performed under 
supervisory controL 

Tireless, never bored, hard to 
overload. 
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Man’s motor output seems to have a 
bandwidth of about 10 cycles per 
second, with a natural periodicity 
(to be avoided) of \ to 1 cycle per 
second. 

The motions possible with the human 
body, though marvelously contrived, 
are limited in amplitude and articu- 
lateness — some motions are im- 
possible, such as telescopic extension of 
limbs. 

Performs well in emergencies. Can take 
remedial measures. Man is adaptable 
and can “reprogram” himself. 

Man is often nonlinear in his manipula- 
tion of controls. 

Humans are highly variable in physique 
and capability (Fig. 31). Allowance 
must be made in interface design for 
this variability. 



A machine can be designed for 
almost any bandwidth if one is 
willing to pay the price. 

In principle, machines are not 
limited in amplitude and articulateness 
of motion. 


Machines do not adapt well to 
emergencies. They either stop or plod 
blindly ahead. 

Linearity or any other function can 
be built into machine controls. 

Machines can be built with fairly 
well standardized interfaces. 



Arm and Foot Movement Limits 


Motion 

Description 

5%ile 

50%ile 

95%ile 

Range 

(95% of sample) 

A 

Foot rotation* 
(side to side) 


78° 

CQ £. 

46°—1 10° 

B 

Arm rotation* 
(side to side) 

| | 

48 

1 1 

30 -66 

C 

Arm rotation* 
(side to side) 

z 1 

134° 

9 , > 

Z CD 

100° -168° 

D 

Arm reach radiust 

27.7” 

30.0” 

32.3” 

No 

E 

Wrist reach radiust 

7.0” 

7.6" 

8.2" 

data 

F 

Elbow reach radiust 

16.5” 

18.0” 

19.4” 

available 


*Air Force personnel, 
tlndustrial workers. 


Figure 31 Some arm and foot movement limits of use in designing 
anthropomorphic controls. 




Table 5 — Man’s Senses As Informational Channels: A Comparison of the Intensity Ranges 
and Intensity Discrimination Abilities of the Senses* 


Oi 

a\ 


Intensity Range 


Intensity Discrimination 


Sense Smallest Detectable Highest Practical 


Relative 


Absolute 


Vision 

2.2 to 5.7 x 10~ 10 

Roughly, the brightness of 
snow in the midday sun, or 
about 10 9 times the 
threshold intensity 

With white light, there are 
about 570 discriminate 
intensity differences in a 
practical range. 

With white light, 3 to 5 
absolutely identifiable 
intensities in a range of 
0.1 to 50 millilamberts. 

Audition 

1 x 10 9 ergs/cm 2 

Roughly, the intensity of the 
sound produced by a jet 

plane with afterburner or 
14 

about 10 times the 
threshold intensity 

At a frequency of 2,000 cps, 
there are approximately 
325 discriminate intensity 
differences. 

With pure tones about 3 
to 5 identifiable steps. 

Mechanical 

vibration 

For a small stimulator on the 
fingertip, average amplitudes 
of 0.00025 mm can be 
detected. 

Varies with size of stimulator, 
portion of body stimulated 
and individual Pain is 
usually encountered about 
40 db above threshold 

In the chest region a broad 
contact vibrator with ampli- 
tude limits between 0.05 mm 
and 0.5 mm provides 15 dis- 
criminate amplitudes 

3 to 5 steps 

Touch 

pressure 

Varies considerably with body 
areas stimulated and the 
type of stimulator. Some 
representative values: 

Ball of thumb— 0.026 ergs 
Fingertips -0.03 7 to 
1.090 ergs 
Arm-0.032 to 
0.113 ergs 

Pain threshold 

Varies enormously for area 
measured, duration of 
stimulus contact and interval 
between presentation of 
standard and comparison 
stimuli 

Unknown 
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Kinesthesis 

Joint movements of 0.2 
degree to 0.7 degree at a 
rate of 10 deg/min can be 
detected. Generally, the 
larger joints are the 
most sensitive 

Unknown 

No data available 

No data available 

Angular 

acceler- 

ation 

Dependent on the type of 
indicator used 

1. Skin and muscle 
senses 1 deg/sec 2 

2. Nystagmic eye move- 
ments 1 deg/ sec 2 

3. Oculogyral illusion 
0.12 deg/sec 2 

Unconsciousness or “black- 
out” occurs for positive 
“g” forces of 50 to 8 g 
lasting 1 second or more 

Negative forces of 3 to 4.5 g 
cause mental confusion, 
“red-vision” and extreme 
headaches lasting some- 
times for hours fol- 
lowing stimulation 

No data available 

No data available 

Linear 

accel- 

eration 

In aircraft -0.02 g for 
accelerative forces and 
0.08 g for decelerative 
forces 

For forces acting in the 
direction of the long 
axis of the body, the same 
limitations as for angular 
acceleration apply. 

No data available 

No data available 


* Adapted from Ref. 55. 
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Table 6 — A Comparison of the Frequency Ranges and Frequency Discrimination 
Abilities of Some of the Senses* 



Wavelength or Frequency Range 

Wavelength or Frequency Discrimination 

Sense 

Lowest 

Highest 

Relative 

Absolute 

Vision 

(hue) 

300 mil 

1,500 m/Li 

At medium intensities there 
are about 128 discriminable 
hues in the spectrum 

12 to 13 hues 

Interrupted 
white light 

Unlimited 

At moderate intensities and 
with a duty cycle of 0.5, 
white light fuses at about 
50 interruptions per 
second 

At moderate intensities and 
with a duty cycle of 0.5, it 
is possible to distinguish 
375 separate rates of 
interruption in the range 
of 1 to 45 interruptions 
per second 

No greater than 5 or 6 
interruption rates can be 
positively identified on 
an absolute basis. 

Audition 
(pure tones) 

20 cps 

20,000 cps 

Between 20 cps and 20,000 
cps at 60 db loudness, there 
are approximately 1,800 
discriminable steps 

4 to 5 tones 

Interrupted 
white noise 

Unlimited 

At moderate intensities 
and with a duty cycle 
of 0.5, interrupted 
white noise fuses at 
about 2,000 interrup- 
tions per second. 

At moderate intensities and 
with a duty cycle of 0.5, 
it is possible to 
distinguish 460 separate 
interruption rates in the 
range of 1 to 45 interrup- 
tions per second. 

Unknown 

Mechanical 

vibration 

Unlimited 

Unknown, but reported to 
be as high as 10,000 cps 
with high intensity stimu- 
lation. 

Between 1 and 320 cps, there 
are 180 discriminable 
frequency steps. 

Unknown 


'Adapted from Ref. 5 5. 
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Table 7 — Characteristics of the Senses* 


Parameter 

Vision i 

Audition 

Touch 

Vestibular 

Sufficient 

Light-radiated electro- 

Sound-vibratory energy, 

Tissue displacement by 

Accelerative forces 

stimulus 

magnetic energy in the 

usually airborne 

physical means 



wavelengths from 400 to 

20 cps to 20,000 cps 

>0 to <400 pulses 

Linear and rotational 


700 m/i (violet to red) 


per second 

accelerations 

Spectral 

120 to 160 steps in wave- 

cps (20 to 1000 

A PP S ~ n i n 


range 

length (hue) varying 

cps) 0.3 percent (above 

pps 



from 1 to 20 mju. 

1000 cps) 



Dynamic 

~90 db (useful range) 

140 db (0 db = 0.0002 

~30 db, 0.01 mm to 

Absolute threshold 

range 

for rods = 0.00001 mL 
to 0.004 mL; cones = 
0.004 mL to 10,000 mL 

dyne/cm 2 ) 

10 mm 

^0.2°/sec/sec 

Amplitude 

resolution 

m 

contrast = ~= 0.015 

0.5 db (1000 cps at 
20 db or above) 

0.15 

ML 10 change in 
acceleration 

L i J 

Acuity 

10 arc minutes 

Temporal acuity (clicks) 

Two-point acuity = 




^0.001 sec 

0.1 mm (tongue) to 
50 mm (back) 


Response for 

~0.1 sec 

~0.01 sec (tone bursts) 

Touches sensed as discrete 

~1 to 2 sec nystagmus 

rate for 



to 20/sec 

may persist to 2 min 

successive 




after rapid changes 

stimuli 

Reaction 

~0.22 sec 

— -0.19 sec 

~0.15 sec (for finger 

in rotation 

time for 



motion, if finger is the 


simple 



one stimulated) 



muscular 

movement 
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Best op- 
erating 
range 

500 to 600 (i (green- 
yellow) 10 to 200 
foot-candles 

300 to 6000 cps 
40 to 80 db 


1 g acceleration di- 
rected head to foot 

Indica- 

1. Spatial orientation 

1. Warning or emer- 

1. Conditions un- 

1. Gross sensing of 

tions for 

required. 

gency signals. 

favorable for both 

acceleration infor- 

use 

2. Spatial scanning 
or search required. 

3. Simultaneous com- 
parisons required. 

4. Multidimensional 
material presented. 

5. High ambient noise 
levels. 

2. Interruption of 
attention required. 

3. Small temporal rela- 
tions important. 

4. Poor ambient lighting. 

5. High vibration or 
g forces present 

vision and audition. 
2. Visual and auditory 
senses. 

mation. 


^Adapted from Ref. 55. 
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DEFINING THE MAN-MACHINE INTERFACE 

Given the attributes of man and machine, how does one draw the 
interface between them? In practice, this question is answered by allocating 
tasks or portions of tasks to each. The type of machine and the job to be 
done are important in determining how much man and how much machine 
will be employed in control. Not too surprisingly, the personal philosophy of 
the designer has something to do with establishing the interface: Some 
engineers believe computers should be brought in for supervisory and preview 
control, others want men in the loop at all times. Finally, no matter how 
carefully control tasks are apportioned between man and machine, the match 
will never be perfect. 

To paraphrase the Biblical quotation: Render unto the machine the things 
that are the machine’s. In the very specific area of teleoperator control, the 
problem of task allocation is rather simple because today’s teleoperator 
normally works with the human operator in full real-time control of all 
activity; that is the operator usually renders nothing to the machine in terms 
of control. Of course, much of the sensing and actuating is done by machine, 
but no direct control functions are carried out unless the operator takes 
himself out of the loop and institutes a subroutine. Excluded from this 
generalization are the many local control loops that exist in any sizeable 
machine.* 

When the first master-slaves were developed in the late 1940s, the human 
operator was essentially in full control of every operation — lifting, moving, 
pouring, manipulating. In fact, the word master-slave is rather contemptuous 
of the machine role in the man-machine partnership. Before long, however, 
drills, saws, hammers, and other tools were being used in the same way man 
uses them. In other words, the human operator began to depend upon the 
machine for laborious “subroutines.” No one thought to call a slave-held 
drilling operation a subroutine, but nevertheless the operator did relinquish 
part of the control task to the machine. Tools have become more and more 
important in the effective application of tele operators; and the most 
important of these tools is the general-purpose digital computer. The 
computer is somewhat like man in the way he thinks, but it is undeniably on 
the other side of the man-machine interface. 

Few systematic objective rationales exist for drawing the man-machine 
interface. Obermayer and Muckier have classified past attempts into five 
categories or, more properly, philosophies: 5 7 

1. Automate wherever the task can be described in sufficient detail for 
engineering design, even though man might do some of these tasks 
better. Under this philosophy, man is assigned poorly defined and 
complicated tasks. Result: poor use of both man and machine. 


^Thermal control loops, for example. 
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2. Follow traditional roles and preferences, wherein man serves as the 
prime controller of vehicle attitude and power (as on aircraft). This 
approach has generally been discredited as man-machine systems have 
become more powerful and complex. 

3. Assume specific human capabilities and limitations and design to make 
the best use of man under these conditions. Usually, this has been 
interpreted to mean that man should be used only as a narrow-band, 
simple amplifier. In teleoperator work, man is obviously much more 
than this, being a strategist and decision-maker as well as a supplier of 
control forces. 

4. Assume a formal mathematical model of man (a human transfer 
function) and design the control system as if man were a completely 
specified servo element. As demonstrated in Chapter 3, this approach is 
hardly applicable to teleoperator design. 

5. Make a direct empirical assault on the system with simulators. Because 
some teleoperators are so complex; viz., Hardiman; this is occasionally 
done. 

None of these five approaches really suffices for teleoperator design; they do 
not take into account man’s most essential attributes — planning, 
decision-making, etc., — all difficult to reduce to equations. 

A systematic, logical approach to defining man’s place in a large 
man-machine complex has been proposed by Serendipity Associates. 5 5 This 
rationale, which is summarized in Table 8, is basically a checklist of 
“activities” that a control system designer must consider in allocating tasks 
between man and machine. It is this allocation of tasks, of course, which 
really defines the man-machine interface. The rather subjective thinking-it-out 
approach of Serendipity Associates infers that the location of this interface 
could depend largely upon who is doing the thinking. For example, a young 
engineer brought up in a computer environment might naturally lean toward 

Table 8 — A Possible Sequence of Activities for Assigning 
Tasks to Man and Machine* 


ACTIVITIES FOR DETERMINING THE OPTIMAL ROLE OF MAN 
Activity 1 

Hypothesize the Potential Basic Role of Man 


Outputs 

1. Statements proposing unique human capabilities to foster system performance. 

2. Statements proposing man-rated core performance to foster system performance. 

a. Sensing 

b. Interpreting 

c. Information Processing 
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d. Decision-Making 

e. Controlling 

f. Monitoring 

g. Information Storage 


Activity 2 

Hypothesize Potential Complementary 
and Support Role of Man 

Outputs 

1. Statements proposing utilitarian human capabilities to foster complementary system 
performance. 

2. A check list of personnel support system needs: 

a. Human Maintenance 

b. Human Monitoring 

c. Life Support 

d. System Protection 


Activity 3 

Review Manned System Solution Feasibility 


Outputs 

1. Statements identifying suspect areas of compatibility between human variables and 
system parameters which would eliminate a manned system solution from further 
consideration. 

2. Statements identifying suspect areas of effectiveness or practicality which would 
eliminate a manned system solution from further consideration. 


Activity 4 

Develop a Preliminary Operator Concept 


Outputs 

1. A summary of operator performance in the local and remote segments for operations, 
maintenance, and support. 

2. Estimates of the number of operators. 

3. Summary of concepts for operator performance achievement and proficiency 
maintenance. 

a. Selection 

b. Training 

c. Job Aids 

d. Human Engineering 

e. Proficiency Maintenance 


Activity 5 

Analyze Personnel Support Requirements 


Outputs 

1. Statements and supporting data of personnel support system requirements and 
development of operational implications. 
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Activity 6 

Review Potential Operator Role for 
Acceptance and Reliability 


Outputs 

1. Summary statements of acceptability of manned system concept indicating areas of 
possible lowered reliability due to unacceptable role requirements. 


Activity 7 

Synthesize Optimal Operator Role 


Outputs 

1. Description of an optimal operator role including the operator concept, personnel 
support requirements, and operator effectiveness. 


ACTIVITIES FOR DETERMINING THE OPTIMAL 
ALLOCATION OF FUNCTION 

Activity 8 

Establish Feasibility of Man-Rated Allocation 


Outputs 

1. Statements and rationale supporting the feasibility or unfeasibility of a man-rated 
allocation for function performance. 


Activity 9 

Develop Potential Man-Rated Allocations 

Outputs 

1. Descriptions of alternative man-rated allocations varying in extent of man 
participation for each function. 


Activity 10 

Review Allocation Poential Against Psychophysical Capacities 


Outputs 

1. Statements and data identifying allocations which are unfeasible because of 
incompatibility with basic psychophysical capacities. 


Activity 1 1 

Review Allocation Potential Against System 
or Function Constraints 


Outputs 

1. Statements and data identifying allocations which are unfeasible because of system 
environmental constraints. 

2. Statements and data identifying allocations which are unfeasible because of system or 
function performance constraints. 
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Activity 12 

Review Allocation Potential Against Human Reliability 

Outputs 

1. Statements and data identifying allocations which are unfeasible because of suspect 
human reliability. 


Activity 13 

Synthesize Man-Rated Allocations 

Outputs 

1. A summary and preferred order of potential man-rated solutions for accomplishment 
of each function. 


* Adapted from Ref. 55. 

mechanizing control tasks that an old-timer would assign to a human 
operator. This ambiguity in task allocation and the location of the 
man-machine interface is also inherent in the long but rather fuzzy list of 
man-machine attributes appearing earlier in this chapter. 

In the context of future teleoperator design, the man-machine interface 
seems to be moving in the direction of letting the machine do more of the 
work. The following general assertions emphasize this trend and also offer 
some general guidelines relative to establishing the man-machine interface in 
teleoperators. 

Assign to Man These Control-Oriented Tasks: 

Pattern recognition 
Target identification 
New, exploratory manipulation 
Long-term memory 

Trouble-shooting, emergency operation 
Hypothesizing, ideation, planning 
Interpreting variable format data 
Inductive thinking 

Setting goals, priorities, evaluating results 

Assign to Machine These Control-Oriented Tasks: 

Monitoring multichannel inputs 

Boring, repetitious manipulation 

Precision motions and precision force applications 

High-speed motions, particularly oscillatory 

Short-term memory 

Computing 

Monitoring 
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Deductive analysis 
Development of optimal strategies 
Non-anthropomorphic motions 

The tendency today is unquestionably to let the machine portion of the 
teleoperator do the hard, repetitious work, while the human thinks, plans, 
and explores. As machines become better able to identify and manipulate 
targets in accordance with man’s general instructions, the machine will take 
over even larger portions of the control task. 

The philosophy of applying machine (computer) control wherever 
reasonable has a profound effect upon the design of displays and control 
hardware. If man is to adopt more and more the function of an executive, he 
will need more executive-type controls; that is, controls that switch in 
subroutines. A specific subroutine could be initiated by a switch, a coded 
signal, or even voice command. Supervisory controls, therefore, are abstract 
and far-removed in terms of spatial correspondence from the master arm of a 
master-slave manipulator. The more “intelligent” the machine, the more 
abstract the controls and the less often man would enter the control loop to 
operate the controls directly. At the far end of the spectrum — where the true 
robots dwell — today’s crude anthropomorphic master arms and hands would 
be replaced by general verbal instructions. 


BRIDGING THE INTERFACE 

Once control tasks have been divided between operator and machine, 
there remains the “communication problem,” which means insuring that man 
can command the machine efficiently and that the machine can feed back 
information to man with ease. The two points of contact where matching is 
necessary are at the displays (machine output) and the controls themselves 
(man’s output) (Fig. 8). 

For effective control of the teleoperator, many engineers believe that the 
controls should be organized like man; that is, be anthropomorphic — a true 
extension of man. This interface-bridging philosophy* is different from, if 
not opposed to, the school that wants to make fuller use of machine 
capabilities and supplants anthropomorphic controls with switches that 
initiate machine-controlled subroutines. Obviously, the more the machine is 
in command, the less anthropomorphic the controls need to be. In hardware, 
these two philosophies are represented by the ANL electric master-slaves with 
slaved TV display on one hand and the largely computer-controlled 
manipulators at Case Western Reserve and M.I.T. on the other. In between are 
a few manipulators displaying various combinations of anthropomorphism 
and the more abstract, symbolic controls. This bifurcation of the field is 
becoming more evident each day. 
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There are undeniable advantages in anthropomorphism and spatial 
correspondence,* the two prime tenents of the make-the-machine-like-man 
school: 

1. An operator can use skills learned in everyday life to run the machine. 

2. Operation is natural, not abstract, and requires less training. 

3. Tasks requiring a high degree of physical coordination are often 
possible; e.g., hula-hooping. 

4. The teleoperator is “generalized,” like the human operator, and is 
readily adaptable to many varied tasks. 

5. The operator “feels at home.” He identifies himself better with the 
task. 

6. It is suspected but not objectively proven that an operator’s reactions 
in emergencies are quicker and more effective. 

The computer-oriented school employs man’s higher powers — planning, 
decision-making, etc: — and matches these abstract outputs to the machine 
using codes understood by the machine. Using today’s technical vernacular, 
the first philosophy matches hardware to the human; the second matches 
software to the human. Some advantages and disadvantages of an abstract, 
software dialog are: 

1 . Abstract language can communicate more control information per unit 
time to the machine. 

2. Nonanthropomorphic commands, such as wrist rotation, can be given. 

. 3. Man is not “wasted” in dull, routine tasks and can use his higher 
faculties to do a better job. 

4. The abstract language is usually highly specialized and may not meet 
the requirements of the task, especially an emergency. 

5. Repetitive tasks can be done with high speed. 

We have discussed so far matching the machine to the human operator; 
perhaps the operator is malleable too. Operator selection and training can 
help bridge the man-machine interface. Operators should be selected with the 
same care as for jet pilots. Factors such as depth perception, eye-hand 
coordination, and reaction time are important in the operation of 
contemporary master-slaves. Good physical condition is also a prime requisite 
because remote manipulation is arduous, demanding work. Training with 
manipulators or simulated tasks is essential. Although a few minutes with a 
master-slave can give a novice a good feel for the machine, only many months 


* Spatial correspondence exists when a motion by the human controller is duplicated 
by the machine. 
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of experience will make a good operator. At the other end of the teleoperator 
spectrum, one would suppose that operating a teleoperator possessing a large 
array of subroutines would require high analytical power and abstract 
reasoning capabilities. But none of these man-and-computer-controlled 
teleoperators has been used operationally as yet. 

Summarizing this chapter, we note that the man-machine interface is at 
best a poorly drawn boundary between man and machine, particularly in 
teleoperators. It is a dynamic boundary that changes with the application, 
with the machine state of the art, and even with the personal philosophy of 
the teleoperator designer. There is no detailed, well-defined, objective 
rationale that tells a designer how to deal with the man-machine interface. 
There are rules-of-thumb, opinions, and checklists for thinking out the 
problem. Basically, the field of teleoperator design is too young for hard and 
fast rules. Despite the lack of rigor evident above, the man-machine interface 
has been surmounted many times in many ways during the past two decades. 
In the next two chapters, we relate some of the solutions — past, present, and 
future — in terms of control and display hardware. 



Chapter 5 


TELEOPERATOR CONTROLS 

AN OVERVIEW 

The fundamental function of switches, joysticks, and other control 
hardware is to translate the commands of the operator into signals that can be 
understood by the machine portion of the teleoperator. In control engineer’s 
language, a control is really a transducer — a device that converts a signal 
from one form to another; for example, the force on a joystick to a 
proportional voltage operating a motor. The signals generated by the control 
hardware may be simply proportional to some physical input from the 
operator, or they may be symbolic; that is, they may contain coded 
meanings, such as move from point A to point B. A simple symbolic input, 
perhaps generated by a typewriter, can release a subroutine containing a long 
train of “primitive,” low-level signals to the teleoperator’s basic actuators. 

Man’s signals to his machines are usually generated by his hands. Direct 
force activates most teleoperator controls, including those that switch in 
subroutines via a teletypewriter. Force and pressure from man’s appendages 
also configure complex controls, such as analog or replica controls, or activate 
arrays of switches in complex patterns. In some cases, particularly in the 
medical field, control forces are created by the feet, the tongue, the head, and 
various muscles throughout the body. Man also generates more subtle 
outputs; eye movements, muscle bulges, and electromyographic signals from 
electrodes attached to the body are used for control purposes. Finally, the 
human voice can carry a heavy traffic of control information if we can find a 
machine that can listen and interpret properly. 

Regardless of how the human body creates control signals, they can be 
classified into four types: 

1. On-off signals, which may simply activate a motor or perhaps begin a 
long, complicated subroutine. 

2. Proportional signals, which might control the speed and direction of a 
motor. 

3. Configuration signals, where an input control device is placed in a 
specific configuration by the human operator. The device then 
generates signals representative of this configuration and the tele- 
operator actuators try to attain the stipulated configuration. Often this 
kind of control is termed position control It is employed in many 
master-slaves, exoskeletons, and walking machines. 

4. Symbolic signals, with intrinsic, coded meaning. 
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Using the above classifications and the various types of physical controls 
associated with teleoperators, we can construct the matrix shown in Table 9. 
Most of the more sophisticated tele operators, it will be noticed, utilize 
configuration (position) control. In most cases, the control configurations are 


Table 9 — Overview of Control Hardware 


Basic 
type of 
control 

Hardware 

manifestations 

Types of tele- 
operators using 
control type 

On-Off 

Various switches (hand, 
eye, voice, muscle 
bulge, EMG) 

Unilateral manipu- 
lators, artificial 
limbs, lunar surface 
samplers 

Proportional 

Potentiometers, joy- 
sticks, voice, 
muscle-bulge devices, 
EMG controls 

Unilateral manipu- 
lators, artificial 
limbs 

Configuration , 

Cables, potentiometers, 
servos (all located 
on the analog) 

Bilateral manipu- 
lators, walking 
machines, exoskeletons 

Symbolic 

Typewriters, voice, 
punched cards, other 
software, switches 

All teleoperators that 
employ supervisory 
control subroutines 


those taken by the human body ; but this is not always true — analog controls 
may assume decidedly non anthropomorphic shapes. 


SWITCHES AND SWITCHBOXES 

The simplest teleoperator control is the on-off switch. Many hundreds of 
unilateral manipulators now working in hot cells and on submersibles are 
controlled from switchboxes and switch consoles. The advantages of the 
switch are many: simplicity, low cost, reliability, no load reflected to tire the 
operator, and switchboxes can be made small and portable, just the thing to 
carry from porthole to porthole within a cramped submersible. There are 
accompanying disadvantages, too. Switches are open-loop controls; there is 
no force feedback. Unless potentiometers or multiple-pole switches are used, 
there is no control over the rate of tele operator joint movement. Only one 
joint can be activated at any instant during precision manipulation. (Some 
fast slewing motions can be carried out using more than one degree of 
freedom.) Lastly, switch arrays bear little resemblance to the manipulator 
configuration; they are decidedly nonanthropomorphic; and operator identi- 
fication with the task is small. Nevertheless, in many applications, simplicity, 
reliability, compactness and low cost win out. 
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On-off switch control may be attained with toggle switches, push-button 
switches, or slide switches. If rate control is also required, a separate 
potentiometer can be connected in series with the switches. Often, however, 
proportional control of manipulator joint movement is achieved by installing 
rotary or linear potentiometers as the primary control elements. Pressure- 
sensitive resistance elements, strain gauges, and piezoelectric elements can 
also provide an output proportional to the force applied by the operator. 
Proportional controls are usually spring-loaded so that they return to a null 
position when the operator removes his hand. Most on-off push-button and 
lever-type switches also return to zero when released. Actually, manipulator 
joint motion is “three-valued;” that is, left /stop /right or counterclockwise 
/stop /clockwise. The corresponding control switches are also three-valued. 
Three-way toggle switches (Fig. 32) are common and so are pairs of 
spring-loaded push buttons (Fig. 33). 



Figure 32 A control box for an underseas electrohydraulic unilateral 
manipulator . Most switches are three-way . (Courtesy of Westinghouse Electric 
Corp.) 
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Figure 33 A push-button control box for an underseas electrohydraulic 
unilateral manipulator. (Courtesy of Electric Boat Division , General Dy- 
namics.) 



Figure 34 The switchbox for the PaR 3000 unilateral electric manipulator. 
Note the three-position switches for some pivots. (Courtesy of PaR.) 
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Switchboxes or control arrays are arrays of on-off switches, potentiom- 
eters, and feedback signals arranged in a convenient, logical fashion (Fig. 34). 
There will be one switch or pair of switches for each teleoperator degree of 
freedom. Switches may be color-coded; coding by shape is also common 
because the operator should keep his eyes on his work rather than the 
switchbox. To build in a little anthropomorphism, three-valued switches are 
connected to move a joint to the right when the switch is moved to the right 
and vice versa; ditto with up-and-down motion and rotary motion. 

Switches and potentiometers usually connect directly with the electric 
motors that drive the joints in all-electric tele operators. In electrohydraulic 
and electropneumatic teleoperators, switch controlled electrical signals open 
and close valves (Fig. 35). When the teleoperator is far away from the control 
station, the control signals may be time-multiplexed, as is common in 
conventional remote control. In space work, the control signals may be 
digitized before transmission, as described below for the Surveyor surface 
sampler. 
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The NASA-JPL Surveyor surface samplers, while not particularly dexter- 
ous teleoperators, did carry out many lunar experiments during which they 
manipulated lunar soil and rocks. In one instance, a surface sampler was used 
to dislodge a Surveyor alpha-scattering experiment which was hung up on the 
spacecraft — a classic example of the use of a tele operator for repair in a 
distant, hostile environment. 



Figure 36 The Surveyor surface sampler in extended position. There are 
only four degrees of freedom . 5 8 


The sampler’s four degrees of freedom were driven by reversible motors 
(Fig. 36) activated by digital commands dispatched from NASA’s Goldstone 
Deep Space Network station in California. The only feedback to the operator 
consisted of television pictures (delayed by the signal transit time of about 
1.3 sec) and telemetry signals indicating the current delivered to the motor 
being operated. Only one motor could be activated at a time — and then only 
in 2- or 0.1-second increments. Except for this quantization of motion, the 
surface sampler operated much like a unilateral manipulator in a terrestrial 
hot cell. 

The sampler controls, however, were not the simple switches we associate 
with unilateral manipulators. The operator had to send digital commands to 
activate the proper motors, as indicated in Table 10. The selection of the 
digital command word is analogous to selecting a switch on a switchbox and 
pressing it. The operator could also select, using additional commands, the 
length of the time the motor would run (2 or 0.1 sec) and the number of 
motion increments allowed. Thus, the operator could watch his television 
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Table 10 — Output Commands from the Surveyor Surface 
Sampler Command Encoder* 


No. 

Digital 

word 

Command 

Destination 

Function 

1 

1101 

(or) 

OlOlf 

Squib-enable 

ON 

Low-power 

latch 

“Arms” or enables squib-firing 
circuit 

2 

mi 

Squib-enable 

OFF 

Low-power 

latch 

“Disarms” or disables squib- 
firing circuit 

3 

0011 

Release 

mechanism 

Squib-firing 

circuit 

Provides power to activate pin- 
puller squib and unstow the 
mechanism 

4 

0000 

Coarse-timing 

mode 

Low-power 

latch 

Sets 2-sec timing mode for 
motors 

5 

0111 

Fine-timing 

mode 

Low-power 

latch 

Sets 0.1-sec timing mode for 
motors 

6 

0001 

Extend 

Motor 

switch 

Provides power to extension- 
retraction motor to extend 
mechanism 

7 

0110 

Retract 

Motor 

switch 

Provides power to extension- 
retraction motor to retract 
mechanism 

8 

1011 

Rotate left 

Motor 

switch 

Provides power to azimuth motor 
to rotate mechanism to the left 

9 

1100 

Rotate right 

Motor 

switch 

Provides power to azimuth motor 
to rotate mechanism to the 
right 

10 

1001 

Open scoop 

Motor 

switch 

Provides power to scoop motor 
to open scoop door 

11 

1110 

Close scoop 

Motor 

switch 

Provides power to scoop motor 
to close scoop door 

12 

0100 

Elevate 

Motor 

switch 

Provides power to elevation 
motor to rotate mechanism 
upward 

13 

0010 

Lower 

Motor 

switch 

Provides power to elevation 
motor to rotate mechanism 
downward 

14 

1000 

Disengage 

clutch 

Solenoid 

switch 

Provides power to elevation 
clutch solenoid to disengage 
elevation drive 

15 

1010 

All motors 
OFF 

Motor solenoid 
control 

Turns off power to motors and 
clutch solenoid 


* Adapted from Ref. 59. 
f Redundant command. 
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Table 11 — Surface Sampler Command Tape 901* 


Minor sequence number 

Surface Sampler command 

3150 

Squib enable 

3151 

Squib enable backup 

3152 

Release mechanism 

3153 

Extend four 2-sec steps 

3154 

Extend five 2-sec steps 

3155 

Retract one 2-see step 

3156 

Lower one 2-sec step 

3157 

Elevate one 2-sec step 

3250 

Lower two 2-sec steps 

3251 

Rotate left two 0.1-sec steps 

3252 

Rotate right eight 0.1-sec steps 

3253 

Lower ten 0.1-sec steps 

3254 

Elevate two 2-sec steps 

3255 

Rotate left one 2-sec step 

3256 

Extend six 2-sec steps 

3257 

Rotate right one 2-sec step 

2450 

Open scoop N 0.1-sec steps 

2451 

Retract two 2-sec steps 

2452 

Close scoop N 0.1-sec steps 


*Ref. 58. 

monitor and proceed stepwise through his experiment using the move-and- 
wait strategy recommended for time-delay situations. 

Preprogrammed tapes were also employed for some sampler motions, 
providing a form of supervisory control. This subject will be covered in more 
detail toward the end of the chapter; a typical series of preprogrammed 
commands used for the surface sampler will suffice here (Table 11). The 
preprogrammed tapes reduced the burden on the operator considerably 
during surface sampler operation. 


JOYSTICKS 

A joystick is a stick-like control that may be tilted forward and backward, 
sideways; it may also be twisted or pushed in and out along its axis. Buttons 
and switches are frequently mounted within reach of the operator’s fingers 
while he is manipulating the stick. A joystick consolidates controls for several 
degrees of freedom into a single piece of hardware. Two important features of 
joystick control are proportionality (joystick displacement or pressure can be 
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employed for rate control of a joint) and directionality (joint motion can be 
reversed simply by reversing the joystick polarity). The joystick illustrated in 
Fig. 37 shows how seven degrees of freedom can be controlled with a single 
joystick, although it is unusual to make joysticks so complex. 


Table 12 — Comparison of Force-Operated and 
Position-Operated Teleoperator Controls* 

Force-Operated (Isometric) Position-Operated (Isotonic) 


Controller output corresponds to forces 
applied by operator; “natural” control 

Controller output drops to zero unless 
manual force is maintained on the 
controller; i.e., it is self-centering 


A large output range may be accurately 
controlled by a small range of control 
lever displacement 
Large manual forces are required 
to control a large output range 
accurately 

Because large manual forces are required 
to control a large output range 
accurately, a controller must be built 
and located so the operator may exert 
large manual forces on it 


* Adapted from Ref. 27. 


Controller output does not correspond 
to forces applied by operator; an 
interpretive step is required for control 
Control lever remains at position last 
set; output remains applied without 
maintaining manual force. (Controller 
usually maintains a set position by virtue 
of sliding friction.) 

To control a large output range 
accurately, a large range of control 
lever movement is needed 
A large output range can be controlled 
accurately with very small manual forces 

Because a large output range can be 
controlled accurately with very small 
manual forces, many types of controls, 
in a large range of locations, may be 
employed. 


Joysticks may be either force-operated (isometric or “stiff-stick”) or 
position-operated (isotonic). Manipulators have been constructed using both 
types. No clearcut advantages have been demonstrated for one over the other. 
Kelley has tabulated the relative advantages and disadvantages of the two 
types (Table 12). 

A joystick is often a better control device than an array of switches 
because the operator identifies better with the task, particularly if the 
joystick is built along anthropomorphic lines like that pictured in Fig. 37. 
Crawford and Kama have compared operator performance on a unilateral 
rate-controlled manipulator using both a joystick and an array of levers. 60 
They found the joystick to be superior. Pesch has compared the joystick 
against a pushbutton array and also found it superior. 6 1 The fact that several 
joints can be controlled by a single, rather anthropomorphic joystick also 
helps make this controller superior to switches and switchboxes. 
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-MOTION TABULATION 

SYMBOL 

MOTION 

A 

SHOULDER ROTATION 

B 

SHOULDER PIVOT 

C 

ELBOW PIVOT 

D 

WRIST ROTATION 

E 

WRIST PIVOT 

F 

WRIST DEFLECTION 

G 

HAND GRIP 


6 MOTION + GRIP MANIPULATOR 


HUMAN ARM MOTION 
FUNCTIONS 




6 MOTION + GRIP 
JOYSTICK CONTROL INPUT 
DEVICE 


Figure 37 Three drawings showing some parallelisms between operator arm , 
manipulator actuators , and a rather sophisticated joystick . Most joystick- 
controlled manipulators are unilateral despite the anthropomorphism shown 
here . 


The motions or pressures on a joystick activate the switches and (more 
commonly) potentiometers which control the joint motors. There is no force 
feedback in the usual joystick, although one can see how the addition of 
servos to the joystick might be accomplished. 

Almost all unilateral manipulators now in operation employ switchbox 
controllers. But a few exceptions exist: one is the venerable General Mills 300 
manipulator still in hot-cell use. Control of this unilateral manipulator is by 
two joysticks (Fig. 38). The General Mills control console is a rather 
formidable piece of equipment, but most joysticks can be made smaller and 
more portable. Westinghouse, for example, has constructed a small joystick 
for controlling the electrohydraulic manipulator arm it built for the Deep Sea 
Rescue Vehicle (DSRV-1). The DSRV joystick has the feature of shifting 
from one degree of freedom to another in much the same way one shifts gears 
in an automobile. In the left-hand position, the operator twists the control 
one way or the other to activate the manipulator shoulder joint; in the 
right-hand position control shifts to the elbow pivot; the forward position 
takes care of the wrist pivot. The back position, however, activates a 
supervisory -control subroutine called True Arm Extension; a straight 
in-and-out motion for scrubbing, sawing, etc. Another rather sophisticated 
joystick built by Electric Boat is illustrated in Fig. 39. 




Figure 38 The console and two joysticks employed in controlling the 
General Mills Model 300 electric unilateral manipulator . The left-hand 
joystick controls the overhead bridge carriage and hoist that positions the 
arm. The right-hand joystick controls all arm and hand motions. (Courtesy of 
R . Karine n f Programmed and Remote Systems.) 



Figure 39 A joystick-type control for an underseas electrohydraulic uni- 
lateral manipulator. (Courtesy of Electric Boat Division, General Dynamics.) 
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Despite the potentialities of joystick control in the tele operator field, 
joystipks are still used sparingly. Switches still predominate in the control of 
unilateral manipulators and, for that matter, artificial limbs. 


ANALOG CONTROLS 

Rather than push buttons or tilt joysticks to maneuver a manipulator arm 
into the desired position, why not make the controller an analog or replica of 
the working arm and design controls that force the working arm to duplicate 
the configuration of the control?* This, of course, is approximately what a 
master-slave manipulator does. Master-slaves are a step more complicated, 
though, because they also provide force feedback. The usual analog control 
displays no force feedback, although it could be built in as in the case of the 
joystick. In fact, an analog control can be thought of as a many-jointed 
joystick, although it is neither isotonic nor isometric. 

Each joint in the analog control master arm has a potentiometer pickoff 
which supplies a signal to the real arm. If the slave arm is in a configuration 
different from that of the control arm, the corresponding signals from 
pickoffs on the slave arm will not correspond to those from the master 
control arm. Joint motors will be driven until all differences are nulled and 
master and slave arm configurations are identical.! The word “configuration” 
is used here intentionally because configurations but not linear motions are 
identical— even if the control arm is considerably smaller than the actual arm 
(a useful feature aboard a crowded submersible). During manipulator 
operation, an arm activated by an analog control may lag significantly behind 
the motion of the master control because joint motors have limited speeds. 
Thus, there is not necessarily good spatial correspondence. 

The advantages of analog control are several: 

1. The controls can be made much smaller (or larger) than the slave 
arm/hand combination. 

2. The controls can be activated from the master-hand area alone, with 
the rest of the joints following the hand motion naturally — like 
railroad cars. Some arms, such as those of Project MAC, are made with 
many more joints than the human arm to facilitate this sort of 
terminus control 

3. Force feedback can be accommodated easily. 


*This is called analog control , replica control model (or model-arm) control, 
position-servo control , and, if the control arm is smaller than the real arm, miniature-arm 
con trol 

f The use of the words “master” and “slave” should not make the reader confuse 
analog-controlled manipulators with master-slaves; master-slaves can be actuated from 
the slave side (they are truly bilateral) but an analog-controlled unilateral manipulator 
cannot 
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4. Preprogramming for supervisory control consists simply of providing 
simulated pickoff voltages. As an alternate, routine motions can be 
accurately carried out using a grooved template; with terminus control 
the grooves are in essence preprogrammed instructions. (Picture the 
operator holding the master terminus like a pencil and following the 
template grooves.) 

There are some very real difficulties, too: 

1. Cost and complexity. 

2. It is difficult to incorporate physically all the necessary circuits and 
electrical components, particularly in miniature master control arms. 

3. With a miniature analog control, small motions are greatly amplified in 
the slave arm. 

4. Considerable friction or an automatic braking system has to be built 
into the analog control arm so that it (and the slave arm) will not 
collapse when the operator releases it. This friction may fatigue the 
operator. 

Almost all of the organizations engaged in undersea manipulator 
development have experimented with miniature analog controls. Generally 
speaking, the experiments have shown that analog control is feasible but that 
the disadvantages just listed outweigh the positive features. Most undersea 
manipulators are still controlled by switchboxes and joysticks. 

Programmed and Remote Systems (PaR) has developed an interesting 
joystick-like control for unilateral manipulators that has some resemblance to 
an analog control (Fig. 40). In the PaR Position Controller the control device 
is a rough model of the actual manipulator arm. Instead of position pickoffs, 
though, the control joints utilize rate pickoffs. Thus, as the controls are 
moved, signals proportional to speed of joint rotation drive the manipulator 
motors (via pulse-counting circuitry) at the same angular rate as the control 
joints. Spatial correspondence {configuration correspondence is more accu- 
rate) may not be maintained because the control arm can be moved faster 
than the drive motors can position the real arm. However, the rate-control 
circuits are activated only when the trigger on the Position Controller is 
pulled; controls can be repositioned to recapture configuration corre- 
spondence when necessary. 


MASTER-SLAVE AND SIMILAR BILATERAL CONTROLS 

The master-slave manipulator concept was pioneered by Argonne Na- 
tional Laboratory (ANL) in the late 1940s, when R. C. Goertz’s group 
developed the first mechanical master-slaves. Later, ANL developed a series of 
electrical master-slaves. These machines are described in the first volume of 
this survey. 1 Master-slaves are bilateral teleoperators in which forces and 
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Figure 40 The PaR Position Controller , a special joystick-type control for a 
unilateral manipulator. This type of control is more anthropomorphic than 
the usual switchbox controls. (Courtesy of R. Karinen, Programmed and 
Remote Systems.) 


torques at the master controls are proportionally reproduced at the slave 
actuators and vice versa. Normally, there are seven degrees of freedom, all of 
which can be controlled simultaneously (Fig. 6). Master-slave controls have 
fingers, shoulder, and wrist joints, which make them considerably more 
anthropomorphic than the switchboxes and joysticks just discussed. Opera- 
tion of master-slaves is natural and the operator easily projects himself into 
the work area. Spatial correspondence also exists. The mechanical master- 
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slaves, particularly the famous ANL-developed Mod-8, are relatively inex- 
pensive, reliable, versatile, and easy-to-operate. They are among the most 
common teleoperators in operational use. 

The seven degrees of freedom in the mechanical master-slaves are 
activated by mechanical linkages that physically tie the controls to the 
actuators. Many of the operator’s motions, say, wrist action, are com- 
municated via metal tapes or cables. In a sense, the mechanical master-slave is 
a rather complicated pantograph, with one-to-one spatial correspondence. It 
differs from the analog controls in the sense that it is completely mechanical 
and possesses force feedback.* 

The master hands of the mechanical master-slaves are the focus of the 
operator’s attention. When he wishes to pick up an object he moves the 
master hand to the object; both master and slave arms accommodate; this is 
much like the terminus control employed with analog controls. It is position 
control as opposed to the rate control employed in most switch and 
joystick-controlled unilateral manipulators. Once in the vicinity of the object, 
the operator makes fine position adjustments and orients the wrist. He then 
grasps the object with the fingers or tongs. Thus, master-slave manipulation 
actually consists of coarse terminus control followed by fine hand adjust- 
ments. 

Master-slave hands are anthropomorphic in that the slave fingers are 
controlled by the human thumb and forefinger, the same digits we use to pick 
up objects in everyday life (Fig. 41). The wrist joint, too, is “natural.” The 
typical master hand also possesses some joystick characteristics. The pistol 
grip is surrounded by switches, buttons, status lights, and levers that add 
versatility to the teleoperator. To illustrate, the manipulator may be locked in 
position so that the operator may leave his station without having master and 
slave collapse under the influence of gravity. Force amplification may be 
introduced between master and slave fingers. Even with these “unnatural” 
side benefits, master-slave controls represent a large step toward anthropo- 
morphic controls. 

The physical configurations of the ANL electrical master-slaves are 
patterned after their mechanical predecessors. The master hands, for example, 
are similar in both the mechanical and electrical species. The major difference 
between the two is that the metal tapes and cables connecting master to slave 
are replaced by servos and electrical signals. The electrical signals may move 
via hardwire or radio. It is this last fact that greatly increases the versatility of 
the electrical master-slave over its mechanical cousin. Master and slave 
stations can be hundreds of feet or millions of miles apart when one dispenses 
with mechanical linkages. Electrical master-slaves therefore can be considered 
for use in outer space or wherever great distance separates master and slave 
stations. Moreover, penetrations in hard-to-seal barriers, such as spaceship or 

*Of course, analog controls are electrical and often miniaturized; but they could be 
mechanical in principle. 
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submersible hulls, are easier to design for electrical wires than moving tapes 
and cables. With the added versatility of the electric master-slaves came 
increased cost and complexity. Because of these factors, electrical master- 
slaves have not yet been widely used. 



Figure 42 The ANL Mark E4A slave-arm schematic . The master-arm displays 
a similar configuration , 62 

In the ANL electrical master-slaves, the operator’s input motions are first 
communicated to rotary drums with position sensors by means of metal-tape 
linkages. Thus, the master controls are similar to the mechanical master slaves 
up to the drums (Fig. 42). On the slave side, the situation is reversed; servo 
motors drive drums and metal tapes that actuate the corresponding degrees of 
freedom. Each of the seven degrees of freedom requires a master servo drive 
unit with two, 60-cycle, low inertia servo motors (Fig. 43). As in all true 
master-slaves, the slave hand and arm can control the master — the real 
meaning of bilateralness. On the slave side, four servos are used. Geared 
synchromotors on each side provide positional information. The servo system 
block diagram for the ANL Mark E4A is presented in Fig. 44. 
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Figure 43 ANL E4A master servo drive unit. 62 


During 1966 and 1967, ANL participated with NASA’s Marshall Space 
Flight Center and their contractor, Ling-Temco-V ought, in a study of 
manipulators for use in orbital spacecraft. 6 3 One of the spacecraft studied 
was the Space Taxi, illustrated in Fig. 45 with two electrical master-slave 
working arms and three less sophisticated docking arms. In this study, ANL 
proposed mechanical master-slaves for early availability and electrical 
master-slaves as the best solution, given adequate development time. 

One of the' basic control problems encountered in this study was the 
operator’s limited working volume, a situation reminiscent of that on small 
submersibles, where switchboxes and joysticks are the common solutions. 
The ANL-recommended tele operator configuration inverted the usual master- 
slave arrangement. The working arms are mounted at the spacecraft bottom, 
below the operator’s feet, giving him an unobstructed view and freeing cabin 
volume for torso and arm movements. Instead of the usual master hand 
control (Fig. 41), a master handle or joystick with a trigger and switches was 
proposed (Fig. 46). The master handle (the analog of the slave hand) would 
be position-controlled, just as in the normal terrestrial master-slave. There is a 
strong resemblance between this control handle, the PaR Position Controller 
(Fig. 40) and the Electric Boat joystick (Fig. 39). The ANL approach, of 
course, offers force feedback — a valuable commodity in manipulation— 
which the others do not. 

The problem of restricted operator volume (Fig. 46) was solved in the 
ANL study by the use of indexing. Indexing involves driving the slave arm 
independently of the master. If the operator cannot reach something because 
he has reached the limit of movement of the master control, he can gain 
additional slave arm motion through the use of indexing motors. As the slave 
moves, the master can be repositioned. The effective working volume of a 
bilateral master-slave can thus be increased by unilateral, switch-controlled 
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Figure 44 Servo block diagram for the ANL E4A electrical master-slave (one 
degree of freedom only). 62 


motors. The indexing control switches are often located on the master hands 
in master-slaves. Obviously, there is some loss of spatial correspondence when 
indexing is employed — the price of expanding operating volume. 

Automatic indexing appeared promising in the ANL space study. 
Whenever the master hand would reach the bounds of the operating envelope, 
index motors would automatically switch on until the master hand was 
operating again in the prescribed volume (Fig. 47). In practice, an override or 
initiate switch could be installed on the master handle to give the operator 
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Figure 45 The ANL-LTV-MSFC Space Taxi master-slave electric manipu- 
lator arrangement. There are two master-slave working arms and three 
docking arms. 6 3 



Figure 46 The minimum indexing volume assumed in the ANL-LTV-MSFC 
space manipulator study; dimensions are in inches . 6 3 
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Figure 47 In the ANL-proposed approach to orbital manipulation , auto- 
matic indexing commences when the master handle reaches the outer bounds 
of a previously established control volume . 63 

control over the otherwise automatic indexing motions. ANL studies 
indicated that the “permissive” initiate switch provided somewhat better 
control. 63 

Several other electrical bilateral manipulators have been built. One of the 
most sophisticated and most interesting was the Handyman electrohydraulic 
bilateral manipulator built by General Electric for the Aircraft Nuclear 
Propulsion program in the 19508.* Handyman, with ten servoed joints in each 
arm-hand combination, was better articulated than the ANL seven-degree -of- 
freedom master-slave. Another departure from the basic ANL master-slave 
configuration was the more faithful paralleling of human joints with control 
joints. As shown in Fig. 48, the Handyman master controls are almost 
exoskeletal, particularly the forearms and hands. Even the hand is articulated. 
In other words, Handyman takes a further step toward anthropomorphism. 
One of the rewards is greater dexterity and more compliance with human 
manipulatory tasks. In fact, one of Handyman’s stocks in trade was its ability 
to twirl hula hoops, something difficult for the standard master-slave 
configuration and next to impossible for unilateral manipulators. Along with 
more joints and greater dexterity went complexity and attendant loss of 
reliability and increased cost. Handyman never saw much operational use, but 
it does stand as a milestone in teleoperator development. 

Handyman was made bilateral by joining in one loop a pair of 
position-error servos, connecting them in tandem so that their error signals 
are made common. 64 This common error signal provided a means for 
reflecting force back to the operator (Fig. 49). Figure 50 illustrates the 
Handyman servo loops for one degree of freedom in more detail. There were 
four separate loops; the upper provided position feedback, the two center 


*For more complete descriptions of these teleoperator systems, see Ref. 1. 
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Figure 48 The Handyman master station. There are ten bilateral servo loops 
in each arm-hand combination. Note the exoskeletal aspects , particularly 
around the forearm and hand. (Courtesy of R. S. Mosher , General Electric 
Co.) 



Figure 49 Servo block diagram for the Handyman electrohydraulic manipu- 
lator built by General Electric . 6 4 







MECHANICAL 
ADJUST 


■ MECHANICAL POSITION 
INPUT BY OPERATOR 


SYNCHRO 

DIFFERENTIAL 


TORQUE 

TRANSMITTER 


(BYPASSED EXCEPT FOR 
THE FIVE HANDMOTIONS) 


(SYNCHRO) 

HYDRAULIC POWER (1500 psi) 



SLAVE 
MECHANICAL 
OUTPUT - 


MICAL f 

r ^H 


LOAD 


■ HYDRAULIC POWER (1100 psi) 

Figure 50 The servo loops for one degree of freedom in the Handyman bilateral electrohy dr aulic manipulator. 6 * 


TELEOPERATOR CONTROLS 















92 


TELEOPERATOR CONTROLS 


measured and compared velocities, and the lower loop provided force 
feedback. 

As the operator generated a position input, his action was opposed by the 
force feedback from the lower loop. If a force greater than the feedback was 
exerted, the resultant mechanical motion was converted into an electrical 
signal by the control transformer. This signal was added to the position signal 
arriving at the summing point from the upper loop. The two signals then 
entered the preamplifier. Introduced at the next summing point was a 
velocity signal resulting from the comparison of master and slave tachom- 
eters. This signal introduced damping if the slave actuator began to move too 
rapidly. The power amplifier next received the signals from the summing 
point. It combined and amplified them, then it transmitted the result into the 
lower feedback loop and the slave servo valve. The servo valve motion caused 
the slave joint to move. 

A potential source of instability in Handyman was the mutual interaction 
between the joints and servo systems. Critical damping prevented this. 
General Electric encountered a similar problem with the Hardiman man 
amplifier (see Chap. 3). 

At the AEC’s Brookhaven National Laboratory (BNL) another electrical 
bilateral manipulator has been developed for use with high energy accelera- 
tors. 65 The BNL approach differs from that used by ANL in the application 
of D.C. servos at the joints themselves instead of at the top of the 
manipulator connected by tapes and cables to the joints. Flatau has claimed 
the following direct advantages: 

1. Better frequency response due to direct coupling of motions. 

2. Complete articulation of all motions, such as continuous rotation of 
the slave joints due to the absence of interconnecting cables. 

3. Simplification due to the absence of metal tapes and cables. 

Naturally, there are disadvantages, too: 

L Inability to use identical servo packages for all joints. 

2. Arms are heavy due to servo equipment installed therein. 

3. Somewhat large brush and hysteresis friction in compact D.C. torque 
motors. 

The final bilateral teleoperator considered here is the all-hydraulic 
Hydroman, developed at Oak Ridge National Laboratory for manipulating 
heavy loads in hot cells. 66 Hydromari’s hydraulic control circuit is portrayed 
in Fig. 51. This manipulator possesses force feedback and employs a 
joystick-like control hand. 

Summarizing, the bilateral master-slaves and other associated bilateral 
manipulators add the dimension of force feedback to manipulation. Except in 
the case of the simple and ubiquitous mechanical master-slaves, the cost of 
mechanizing force feedback in terms of money, complexity, and reliability 
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Figure 51 The hydraulic control schematic for the Oak Ridge Hydroman 
heavy-duty manipulator . 6 6 


has militated against widespread application. The ANL electrical master-slaves 
have proven highly versatile and effective in hot-cell work, but still they have 
not been adopted for underseas work — a most logical application from many 
standpoints. When manipulation at great distance is contemplated, as in the 
space program, there is no alternative to electrical master-slaves if force 
feedback is considered essential. However, even here there is a catch: the 
greater the distance separating the master and slave stations, the greater the 
time delay in force feedback. While electrical master-slaves may be useful in 
orbital work, they may be far less attractive on the Moon because an 
Earth-based operator will not feel the reaction forces for over one second. 
However, teleoperators controlled from a manned lunar lander would be very 
useful in reconnoitering the Moon; time delay would be negligible here. 


WALKING MACHINE CONTROL 

If good hand-arm teleoperators can be manufactured, why not leg-foot 
teleoperators; that is, a walking machine, a pedipulator rather than a 
manipulator? Walking machines can be made with ease; a great many of them 
have been constructed over the ages, from small walking toys to huge 
drag-line machines used in mining work. All of these machines have one thing 
in common; they are preprogrammed. Being preprogrammed they are 
completely deterministic, treading ahead blindly regardless of the terrain or 
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obstacles. Even the more modem and sophisticated experimental walking 
machines built by Shigley and Space/General (see Ref. 1) permit the operator 
to do no more than start, stop, and steer. For good off-road mobility, 
however, we need either a highly adaptive, operator-less control system or a 
human operator to provide the adaptation to terrain and obstacles in person. 

In Chapter 3, we mentioned the digital, adaptive control scheme 
suggested by Hoch, at Battelle— Northwest Laboratories. In this scheme, 
adaptation to terrain is accomplished by the analysis of feedback signals from 
the joints of the walking machine. Because the operator would merely drive 
the propelled vehicle, the machine would not be a teleoperator as defined in 
this book. To qualify as a teleoperator, a walking machine must have man in 
the control loop, although he might initiate certain walking subroutines, 
particularly on easy, relatively flat terrain and when operating on prepared 
surfaces. 

If man is to be an intimate part of the control loop, the first impulse is to 
build the walking machine like man; that is, a biped. Instead of controlling 
teleoperator arms, man would control legs. R. S. Mosher, at General Electric, 
has suggested such two-legged pedipulators. General Electric is now develop- 
ing a man amplifier under DOD sponsorship which is essentially an 
exoskeletal biped walking machine with servoed arms. However, the purpose 
of this machine, which is called Hardiman, is man amplification, not off-road 
mobility, and we reserve discussion of this machine until the next section. 

There seems little argument that a biped pedipulator would work if 
carefully controlled by man. General Electric has actually built a Pedipulator 
Balance Demonstrator that has proven that man can balance himself easily 
atop a two-legged servoed machine, 67 In operating this balancing machine, 
the operator’s head is some fifteen feet from the floor and there is a natural 
fear of falling. Nevertheless, operators quickly learn to rely on their senses of 
balance and control the machine successfully. From the neuromuscular 
standpoint, a neophyte operator “knows” how to operate the machine 
immediately — the GE balance machine is that anthropomorphic. 

Despite the success of the Pedipulator Balance Demonstrator, Bradley and 
others have pointed out that a biped walking machine can still fall, just as a 
man does on occasion, then the machine would be out of commission until a 
crane came along to right it. 68 For this and several other reasons not 
associated with control, development interest has now focused on quadruped 
walking machines. In this kind of teleoperator, the human operator controls 
one pair of legs with his legs and the other set with his arms — more or less as 
if he were crawling. Objections to the four-footed walking machine concept 
have been raised by engineers at the Army’s Rock Island Arsenal. 69 Briefly, 
this critique asserts that walking stability in a quadruped is a strong function 
of its active torso. For example, no gait can be found that does not call for 
lifting a leg on a heavy corner; unless the animal’s torso helps shift the center 
of gravity there will be a fall. Since the quadruped walking machine does not 
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Figure 52 Artist's concept of the Walking Truck. (Courtesy of General 
Electric Co ,) 


possess a controlled, articulated torso or other means of shifting balance, this 
critique claims that instability is likely. The Rock Island Arsenal report 
concludes that hexapeds or octapeds — controlled automatically — would be 
more reasonable engineering solutions to off-road mobility. 

The quadruped concept is being tested in a General Electric development 
program sponsored jointly by the Army Tank-Automotive Command and the 
Advanced Research Projects Agency. The objectives of the Walking Truck or 
Quadruped program are to design, construct, and test a full-scale, four-legged 
walking machine capable of carrying an operator and 500 pounds of cargo 
(Fig. 52). Each leg of the Walking Truck has three joints powered by 
force-reflecting hydraulic servos. 

A major requirement in the Walking Truck program is the development of 
effective operator controls. A full-scale simulator was built to test out ideas 
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Figure 53 A quadruped simulator . The simulator is mechanically linked to 
the operator's motions. (Courtesy of General Electric Co.) 


(Fig. 53). The simulator was unpowered but the controls were mechanically 
connected to the truck legs to provide force feedback and position spatial 
correspondence. During operation, the simulator was suspended by a crane, 
and the operator executed walking and turning maneuvers. Human factors 
analysis of simulator tests indicated that satisfactory control of all leg 
motions could be accomplished by a single operator. The simulator, of 
course, could not check out the assertion of the Rock Island Arsenal 
engineers that the machine would fall over in practice. 

General Electric has constructed some prototype hardware for the 
Walking Truck (Fig. 54). The operator sits in a seat wearing exoskeletal 
controls around all four limbs (Fig. 55). A master control arm is illustrated in 
Fig. 56. Metal cables transmit control forces to hydraulic servo-valve 
actuators. Force is reflected hydraulically from each joint to the correspond- 
ing master joint where it is felt by the operator. There is exact kinematic 
symmetry between the master controls and the actual legs. The controls can 
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Figure 54 Photograph of the Walking Truck prototype developed by GE for 
DOD. (Courtesy of General Electric Co.) 


be adjusted to fit different operators. In addition, the force levels required for 
control can be adjusted for best agility and least fatigue. The instability 
predicted by Rock Island Arsenal engineers has not been encountered in this 
design. The Walking Truck, a major advance in the teleoperator field, boasts 
twelve servoed, force-reflecting joints. It is an intimate combination of 
machine and man-as-a- whole. 
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MAN-AMPLIFIER CONTROL 

A man amplifier is an exoskeletal teleoperator that greatly increases the 
physical strength of the operator wearing the structure. The artist’s concept 
of the General Electric Hardiman readily shows the marriage of the Walking 
Truck legs (Fig. 55) with the Handyman arms (Fig. 48). The result is a 
machine envelope for man, with many but not nearly all of man’s 
articulations copied with bilateral servos. The design and applications of man 
amplifiers are discussed in more detail in Ref. 1. 

Although Cornell Aeronautical Laboratory did considerable exploratory 
work on man amplifiers in the early 1960s, 71 the more recent General 
Electric Hardiman project* is the only effort that has attacked the hardware 
problems in depth. 70,72 Hardiman, with fifteen degrees of freedom (some in 
series), is considerably more complicated than even the Walking Truck. The 
project should be classified as “exploratory hardware development.” 

In the Hardiman concept, the operator stands inside an anthropomorphic 
structure built in two halves that are joined together only at the hips by a 
transverse member called the “girdle.” The exoskeleton parallels the operator 
everywhere save at the forearms, where the exoskeleton completely surrounds 
the operator, and his arms are colinear rather than parallel with the 
exoskeleton forearms. This forearm arrangement simplifies controls and 
makes it easier for the operator to identify his arm with the slave arm. The 
slave hand consists of one servoed degree of freedom that forces an opposed 
“thumb” toward a V-shaped palm-finger structure. An additional thumb-tip 
joint is not servoed but responds to an operator on-off switch control. 

The force ratio contemplated between master and slave structures is 
about 25. This immediately raises a question of operator safety should the 
slave exoskeleton somehow run amok. In the GE design, limbs are physically 
linked in such a way that small master-slave errors cannot build up to do 
damage. Another safety feature locks all actuators should hydraulic pressures 
or control signals fail. Collapse of a heavy exoskeleton — carrying perhaps a 
2,000-pound load — would be very hazardous without such a provision. 

The articulation and dimensions of the GE man-amplifier were de- 
termined by a study of the motions that it could perform and the range of 
individual operators that it could accommodate without major adjustments. 
Operators were assumed to range from the 10th to the 90th percentile in 
physical size. Ultimately, the degrees of freedom and dimensions illustrated in 
Fig. 57 were selected for each side of the master-slave. With 15 joints on each 
side, a man-amplifier could carry out most of the important human motions, 
save for those requiring considerable dexterity of the hand. 

In the original Hardiman concept, the operator exerted a force against the 
closely fitting control surface at any particular degree of freedom. The 


*The Hardiman project is sponsored jointly by the Army and Navy. 



100 


TELEOPERATOR CONTROLS 


10 UPPER ARM ROTATE 



2 FOOT ROTATE 
2A ANKLE INVERSION 



POINT OF OPERATOR 
ATTACHMENT TO 

x.} 

MASTER 

, ROTATION 

><J 

AXES 


Figure 57 Isometric stick figure showing the kinematic design of one half of 
the Hardiman exoskeleton . There are 15 degrees of freedom, 10 


surface then moved relative to the encasing slave member and, in doing so, 
actuated a valve in the master control circuit. Several schemes were proposed 
for translating the operator movements into signals that would actuate the 
hydraulically powered slave joints. One was a simple “tickler” or finger 
connected directly to the hydraulic valve (Figs. 58 and 59). Tickler control 
was found to be unsatisfactory for the man-amplifier legs; and control of the 
joint angles was proposed for some leg degrees of freedom (Fig. 60). 

By early 1968 the mechanical design of Hardiman-I had progressed to the 
point where it was evident that a machine housing a human controller could 
be built that could lift and manipulate 1500 pounds. Mechanical-hydraulic 
bilateral servo development, however, had not progressed as rapidly as 
General Electric had expected. The key development problem concerned the 
stabilization of three or more servoed joints in series. (See Chap. 3 for 
theory.) The Walking Truck and Handyman programs had proven that servo 
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Figure 60 Schematic of an exoskeleton servo linkage . 70 
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cascading was possible to a degree. But the Hardiman control requirements 
were so much more demanding that instability was likely using mechanical- 
hydraulic servos. General Electric, therefore, recommended replacing some 
mechanical-hydraulic servos with electrical-hydraulic servos because the latter 
can be stabilized rather easily using electrical circuits. 7 0 * Hardiman design is 
now proceeding on the basis of this change. 


EYE SWITCHES AND OCULOMETERS 

Although teleoperators are primarily manipulatory machines and nor- 
mally should be controlled with the corresponding human extremities, there 
is no a priori barrier to the use of other parts of the operator’s body for 
special control tasks. Man has no prehensile trunk or tail, but his eyes are 
remarkably well-controlled and, as we shall see shortly, his voice can be rich 
in symbolic commands. 

It is difficult to mechanically harness the eye and derive control 
information from its motion. Optical pick-offs, however, have been developed 
for switching, gun aiming, and other purposes. Just how much of this 
technology is applicable to the tele operator field? 

NASA has developed an eye switch (Fig. 61) that depends upon the 
marked difference between the infrared reflection coefficient of the iris and 



Figure 61 Eyeball-controlled electrical switch. As the eye is voluntarily 
moved in the direction of the infrared sensor and light source , the eye's 
higher infrared reflectivity increases the sensor output sharply. (Adapted 
from NASA Tech Brief 65-10079.) 


*This reference contains extensive descriptions and hardware drawings of the original 
Hardiman approach. 
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the area surrounding it. The wearer-operator can voluntarily switch equip- 
ment on and off by directing his eye toward the infrared light source. As his 
eye moves, the infrared sensor mounted on the glasses frame detects the 
change in reflectivity and a switch is thereby closed or opened. If an 
operator’s hands, feet and digits are busy with other controls, an eye switch 
may prove useful, especially in close, demanding manipulation. Auxiliary 
switches of this type are generally actuated by the thumb or fingers in extant 
teleoperators (see the illustrations of joysticks and master control hands 
earlier in this chapter). It would be unusual for an operator’s extremities to 
be entirely saturated with the control task, but an eye switch might be 
convenient if an operator could not turn his head to look at a switchbox. 

Honeywell and other organizations have designed and built eye-controls 
(called oculometers) that permit continuous control of machines. 73 In the 
oculometer the eye is illuminated with collimated light that is reflected by 
the cornea. The position of this reflection, relative to the center of the pupil, 
is proportional to eye direction. To obtain a control signal, the pupil area of 
the eye is imaged into the photocathode of an image dissector tube (Fig. 62). 
The pupil-iris boundary and the corneal reflection are acquired and tracked. 
The eye-direction control signal can be computed by comparing the relative 
positions of the pupil and the corneal reflection. 

The oculometer developed by Honeywell under NASA contract is 
telescopic in configuration; that is, the operator must look through a 
telescope at a target. A helmet-mounted oculometer, though, is quite feasible; 
a possible configuration is shown in Fig. 63. With a helmet-mounted model, 
the operator retains normal, naked-eye vision and is free to move his head. 
The output signal would be proportional to the angle between the operator’s 
eye axis and the forward axis of his head. 

The primary applications of oculometers are in visual search, tracking, 
and instrument pointing. Conceivably, oculometer signals could steer walking 
machines and perhaps point sensors in the operating space, say, a television 
camera. In the next chapter, we shall see how head controls (not 
oculometers) have been employed to visually immobilize a TV scene in 
remote operations. An oculometer could also control TV cameras so that the 
operator sees on his TV monitor what his eyes would see if he were located in 
the working area.* Oculometers may also be combined with computer 
control of tele operators. Man is an excellent identifier of objects and analyzer 
of situations existing in the working environment; the operator might merely 
look at a desired object and give the computer a signal to pick it up or 
perform some other operation on it. In other words, the oculometer can 
pinpoint the direction of a target for the teleoperator. Of course, intense 


*Both the head and the eyes are used to select the scene viewed by the human eyes. 
Any eye-control system must distinguish between these two motions, say, by 
immobilizing the head. 
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Figure 62 Image of the eye on the photocathode of an oculometer. The 
relation of the pupil/iris boundary to the corneal reflection determines the 
direction in which the eye points. 13 


optical concentration is demanded if the operator is not to be distracted by 
outside visual cues. 


VOICE CONTROLS 

Another control signal, the human voice, can convey a great deal of 
abstract control information to the machine. For example, if a voice control 
were combined with the oculometer discussed above, the operator would 
merely look at an object and verbally command the machine to pick it 
up, turn it over, or move to the spot where his gaze is next fixed. 

The most primitive kind of voice control depends only upon the presence 
of sound — any sound — to activate a switch. Such voice switches do not 
discriminate between natural noises and commands from someone other than 
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the operator. Somewhat more selective are voice switches that depend upon a 
certain tone, perhaps a whistle. When the control system can discriminate 
between different tones (whistles, again) the operator could actuate several 
different switches or even continuously control the setting of a control. The 
grip force of a manipulator hand, for example, might be made proportional to 
the frequency of a whistle. Voice controls of this type have not been applied 
to teleoperators, although a few experimental devices have been constructed 
to help handicapped persons. 

A single word in the human language can convey much more than an 
on-off switch command. Consider the fact that one human can verbally direct 
another to carry out the most complex task — in fact, any task that might be 
assigned to a tele operator. Why, then, cannot a human operator verbally 
direct the machine portion of a tele operator to carry out any manipulatory 
task he has in mind? In terms of tomorrow’s technology, he probably can; 
but today’s machines can comprehend only the simplest spoken words. Once 
they understand the words in a command, though, they can carry out the 
command to the letter. To illustrate, it is not too difficult to build a machine 
that can comprehend and act upon verbal “stop” and “go” commands. 
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Stanford Research Institute, Mi.T., and several other organizations are 
exploring the technology of machines that understand the spoken word. 
Sheridan’s group at M.I.T. is the only one currently applying this approach to 
manipulator control. 1 3 The basic manipulator commands are quite simple; at 
least at the lowest, most primitive level; viz., move wrist clockwise, close 
hand, etc. If this were not so, manipulators could not be controlled by simple 
switchboxes. The simple command “pick up,” however, contains more 
information than that intrinsic in the flicking of a single switch. To pick an 
object up with a unilateral manipulator, several switches and their corre- 
sponding degrees of freedom may have to be activated. The human language 
moves easily from simple to complex commands, and today’s machines are 
ready learners. The situation is analogous to the historical progression from 
the early machine-language programming of computers to the more and more 
abstract human-oriented instructions of Fortran and its descendants. 

The first problem in voice control is, of course, speech recognition or the 
identification by the machine of the spoken word; that is, correlation of a 
train of sound patterns with known words in its memory. Once this 
association can be made, the computer can take over. The machine 
recognition of voice patterns is not within the scope of this survey. The 
reader should refer to the literature on the subject. 74 ’ 75 

At M.I.T. an English-language-controlled manipulator is being built using 
a cascade of three processes: 

1. A sentence parser, which recognizes typed (not spoken) words and 
casts them into categories, such as named objects, goals, specific 
actions, adverbs, etc. 

2. A semantic interpreter which operates on the structured statement so 
that it can “understand,” i.e., decide upon unique subgoals. 

3. A manipulation interpreter, which, given the understood subgoal, 
decides upon a sequence of primitive manipulator actions to achieve 
that subgoal. This process may make use of state-space algorithms, or 
heuristic techniques, such as those mentioned in Chapter 3. 

Control of manipulation by English commands is in an embryonic state; 
no hardware has been constructed yet, although the reader will find many 
speech recognition devices described in the literature. 


SPECIAL CONTROLS USED IN 
PROSTHETICS AND ORTHOTICS 

An artificial limb is much like a manipulator but the amputee who 
operates it is at a great disadvantage because he has either lost all or part of 
the analogous flesh-and-blood limb. The amputee often has recourse to his 
remaining hand for actuating controls or he may employ his shoulder, his 



TELEOPERATOR CONTROLS 


107 


feet, and other muscles. Even a person who is almost totally disabled can 
manage to move his head or tongue or some portion of his body to initiate 
externally powered aids. 

If the artificial limb is externally powered, perhaps by batteries or 
compressed gas, switches located somewhere on the body are the most 
common sources of control signals. Switches are simple, cheap, and 
reliable — as mentioned at the beginning of this chapter. They are also very 
limited in flexibility and proportional control is impossible. However, an 
artificial limb can be controlled rather well with a few simple switches. A 
joint on a prosthesis is essentially a four-state device, just one step more 
complex than the ordinary fixed-rate unilateral manipulator joint. According 
to Tomovic, the four possible states are: (1) locked, (2) increasing, (3) 
decreasing, and (4) free. 76 The final state is the one not found on ordinary 
unilateral manipulators. Proportional control is desirable in an artificial limb 
to improve manipulation and make its motion appear more natural. However, 
the simplest artificial limbs are generally the most successful because the 
ordinary wearer does not wish to be burdened with additional complexities. 

The classical way to control artificial limbs is through shoulder shrugs and 
other bodily motions that pull cables attached directly to the artificial limb. 
The following general discussion applies to both self- and externally powered 
devices. 

McLaurin has reviewed the different approaches to musculoskeletal 
control in prosthetics and orthotics. 7 7 He classifies the control motions into 
three groups: 

1. The motion of one part of the body relative to another; for example, 
shoulder elevation, chest expansion, chin movement, humeral flexion, 
elbow flexion, finger motion, and many, many more. 

2. The motion of one part of the body with respect to a fixed object; for 
example, head motion relative to a wheelchair, torso motion (joystick- 
fashion) relative to a chair, and, of course, the eye controls introduced 
earlier. 

3. The motion of parts of the body relative to space; for example, head 
motion relative to local gravity and head motion that causes gyros to 
generate a control signal. 

Many harnesses and special cables have been devised to help an amputee 
control and actuate an artificial limb without external power. 78 The figure-8 
harness shown in Fig. 64 represents an example. The cable that runs from the 
harness down to the arm is called a Bowden cable. On occasion, the control 
cables are surgically connected to the wearer’s muscles in an operation called 
“cineplasty.” 

Wearer-actuated artificial limbs have been in use for centuries. But now 
that compact sources of power have been developed, interest has turned to 
the so-called externally powered artificial limbs and orthotic devices. The 
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most common sources of power are electrical, pneumatic, and hydraulic. If 
the wearer of the prosthesis has one good arm, the requisite switches or valves 
may be located in his pocket or attached to his body where they are readily 
accessible. Switches have sometimes been located in the shoe, when the 
wearer did not want to be too obvious in controlling his prosthesis. 




ABOVE-ELBOW "FIGURE 8" HARNESS 

Figure 64 An above-elbow artificial arm with a hook-type hand. A 
“figure 8” harness is used here. (Courtesy of E. Murphy , U. S. Veterans 
Administration.) 


Muscle-bulge switches are also employed, but like the shoe switches these 
control only one degree of freedom or one speed unless logic circuitry is 
added that translates coded switch signals into more sophisticated motion; 
i.e., two pulses, slow; three pulses, fast; etc. Each controllable degree of 
freedom might have a digital address; three- or four-level commands might be 
transmitted, too. But such codes are generally too much trouble. A few 
simple on-off switches are the rule. 

When both arms are shrunken and deformed, as they are in many 
thalidomide cases, special switchboxes can be installed where they can be 
manipulated by the deformed limb and hand (the phocomelic digits). 79 
Figure 65 shows a five-year-old thalidomide child with two artificial 
electrically actuated limbs built by Northern Electric Company, Ltd., in 
Ottawa. Later, the electric motors were replaced by a hydraulic actuator in a 
search for lighter weight and smoother operation. 80 Control for the hydraulic 
Northern Electric arm is provided by a lever-type selector valve that activates 
one of eight degrees of freedom, plus a switch that controls the direction of 
motion (Figs. 66 and 67). The Northern Electric hydraulic arms have been 
very successful. One patient can even write crudely with the prosthesis. 
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Figure 65 The Northern Electric electric arm. Note the switches actuated by 
the fingers of the subject. (Courtesy of Northern Electric Co., Ltd.) 



Figure 66 The eight-way control-valve select lever employed on the North- 
ern Electric hydraulic arm. (Courtesy of Northern Electric Co., Ltd.) 
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Figure 67 Control system for the Northern Electric hydraulic arm. 
(Courtesy of Northern Electric Co ., Ltd.) 


The Northern Electric controls described above are representative only. A 
large literature exists that describes many different varieties of switches for 
prosthesis control. 8 1 

When the hands cannot be employed at all for switch control, the tongue 
turns out to be surprisingly responsive and effective. Many different types 
have been built. Figure 68 portrays a seven-lever tongue control built by 
Rancho Los Amigos Hospital for high-level paralytics confined to wheel- 






Figure 68 A tongue switch. Each “toggle” is a three-way switch. (Courtesy 
of J. Allen , Rancho Los Amigos Hospital) 

chairs. 8 2 Each switch has three positions. Much more elaborate joystick-like 
tongue-control devices have been designed wherein the tongue “manipulates” 
various levers and buttons. Proportional controls have been built into some of 
these devices. 

Proportional or rate control can be provided for artificial limbs through 
pressure-sensitive devices, such as strain gauges in the teeth and even blowing 
and sucking controls. Although rate controls run a poor second in popularity 
to on-off and three-level switch controls, they do emphasize the remarkable 
flexibility and adaptability of parts of the human body to help the hands and 
arms in the control of complex machines like teleoperators. 

The switches and proportional controls activated by muscle bulges are of 
three basic types: carbon, photoelectric, and strain-gauge. 83 The carbon 
transducers operate on the same principle as telephone transmitters; carbon 
granules are sandwiched between two electrodes. Muscle pressure on the 
electrodes will decrease the electrical resistance between the electrodes. 
Photoelectric transducers can be made in several configurations, one of which 
is illustrated in Fig. 69. The fundamental idea here is to reduce (or increase) 
the quantity of light received by the photocell as the actuating muscle is 
flexed. Strain gauges (Fig. 70) can be attached to muscles to yield a signal 
roughly proportional to the muscle bulge. 

Except for the artificial arms that are controlled directly by cables 
connected to body harnesses or actual muscles, there is little or no force 
feedback present in the schemes discussed above. Neither is there an 
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Figure 70 A strain gauge transducer attached at an abdominal site. (Courtesy 
of J. Lyman , University of California at Los Angeles.) 

analogous limb that would be able to interpret the feedback in most cases. 
This is a severe handicap because objects often fall from the grasp or are 
perhaps broken when the prosthesis wearer cannot feel the force he exerts. 
This deficiency can be compensated for to some extent by building a 
closed-loop control circuit that bypasses the operator. Salisbury et al, at 
Walter Reed Army Medical Center, have installed piezoelectric sensors in the 
fingers of an artificial hand. These sensors detect the vibrations created when 
two surfaces slide over one another. Slippage noises are converted into 
commands that cause the hand grip to increase until slippage stops. 
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These examples represent only a sampling of the control technology that 
has built up during the years around artificial limbs and orthotic devices. 
Although these prosthetic and orthotic devices are included with manipu- 
lators, walking machines, and exoskeletons under the definition of the 
teleoperator, there has been relatively little intercommunication. 


ELECTROMYOGRAPHIC (EMG) CONTROL 

Muscle activity is basically electrical in nature. When electrodes are 
attached near or in any of man’s striated muscles, muscle flexure generates 
electrical signals we can pick off for control purposes. These signals are 
variously termed electromyographic (EMG), or myographic, or muscle-action 
potentials (MAPs). 

According to Alter, the control potential of EMG signals was recognized 
by Norbert Wiener in the early 1950s. 84 The suggestion was quickly followed 
up in Britain and Russia, where prototype EMG-controlled prostheses were 
constructed prior to 1960. The United States has generally lagged behind in 
this field, although Alter’s bibliography demonstrates that American interest 
in EMG controls has increased rapidly of late. 

Most of the work described below was carried out with the application to 
prosthetic and orthotic devices in mind. However, normal people generate 
EMG signals, and these may eventually be employed for controlling other 
kinds of teleoperators. Tiny electrodes, for example, may turn out to be 
much smaller and more comfortable to use than the controls described in the 
preceding sections. One can even visualize gloves or tightly fitting jackets, 
even space suits, with built-in electrodes that an operator would don to 
control a teleoperator with many degrees of freedom. In this concept, the 
motions of the operator would be faithfully duplicated by the actuators, 
located perhaps in a hot cell or on the Moon. Such visions are far off, 
however, for the EMG state of the art is still rather primitive. 

There are three classes of electrodes which may be used to pick up EMG 
signals: skin-surface types, types which pierce the skin, and types completely 
implanted in the body. 8 5 Surface electrodes are obviously the easiest to install 
and remove (Fig. 71). Their disadvantages include weak signals — due to the 
high impedance of the skin, which can be reduced somewhat by electrode 
pastes — and the surface electrode’s tendency to shift, producing “artifacts”; 
that is, unwanted electrical disturbances. There is also “crosstalk” from 
nearby muscles. Electrodes that pierce the skin may be placed just below the 
skin (subcutaneous) or they may penetrate the muscle itself. The skin 
impedance problem is bypassed by this kind of electrode. The intramuscular 
electrodes can pick off signals from different muscles or even from different 
parts of the same muscle. Electrodes that penetrate the skin still have a 
tendency to wander a bit; they may break off, too, leaving a piece of metal in 
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the operator. Further, the electrode site is a source of irritation and potential 
infection. Surgically or hypodermically implanted electrodes that will survive 
the bath of corrosive body fluids and not irritate the operator are difficult to 
design. Surface and skin-piercing electrodes have been employed most 
frequently in EMG work. 

All of the striated muscles are potential sites for control electrodes. In the 
case of the normal person, the muscle selected would ordinarily be analogous 
to the motor driving the same degree freedom in the teleoperator; the biceps, 
for example, might control a manipulator elbow joint. But almost any muscle 
can be trained for EMG-control purposes. Shoulder muscles are used for 
controlling artificial arms in cases where the natural muscle site no longer 
exists.* The teleoperator designer of the future may wish to seize upon this 
attribute for the control of nonanthropomorphic degrees of freedom; say, the 
control of wrist extension via a shoulder-muscle electrode. Even more 
exciting is the discovery that a human operator can voluntarily control single 
motor units in a muscle, a fact that potentially increases the number of 
available EMG control sites by a large factor. In other words, the number of 
output signals under conscious, voluntary control of the operator can be 
many times greater than the number of physical degrees of freedom. Despite 


*In controlling orthotic devices, the natural muscle may still produce useful EMG 
signals even though the real arm cannot be moved voluntarily. 


Figure 71 Surface electrodes for detecting EMG signals. (Courtesy of 
W. Waring, Rancho Los Amigos Hospital) 
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Figure 72 Recordings of EMG signals from a normal subject. 

these promises of future enhanced control through EMG, contemporary 
development programs are oriented toward making simpler systems work 
well, especially those destined for handicapped persons. 

Let us look more closely at the EMG signals; when a muscle is flexed, 
electrodes nearby or embedded in the muscle itself record the summation of 
separate fiber action potentials. 86 In this sense, an EMG signal is an 
“interference pattern” resulting from the addition of numerous signals from 
separate fibers. The observed signal obviously depends upon the location of 
the electrodes. In spite of all the variables, normal muscles produce 
characteristic signal patterns (Fig. 72). Three parameters describe these 
signals: amplitude, spike width, and spike frequency. Amplitudes are usually 
less than 50 millivolts peak-to-peak; while the spike width is measured in 
milliseconds. Spike frequency or repetition rate varies greatly with the muscle 
selected, as indicated in Fig. 72. A plot of power vs. frequency (the signal 
spectrum) is shown in Fig. 73 for two different muscle loads. It is the change 
in signal amplitude with muscle load that allows us to provide proportional 
EMG control to teleoperators. As muscle contraction increases, there is also 
an overall increase in repetition rate ; another potential source of control data. 
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Figure 73 Typical power spectra for surface-electrode EMG signals. 


Superficially, EMG signals would sound almost ideal for teleoperator 
control. EMG signals are certainly more convenient than, say, tongue switches 
for a handicapped person. They are also lighter and more comfortable than 
restrictive harnesses. Furthermore, reasonably accurate proportional control 
has been demonstrated. On the other hand, electric razors and other 
applicances may seriously interfere with EMG signals as does the crosstalk 
from other muscles. Some wearers of EMG-controlled devices feel that EMG 
offers less “positive” control than switches. Lyman, who has made a 
systematic study of the performance of EMG systems in skilled manual 
control tasks, found that the operators were easily fatigued and that 
considerable concentration was required, particularly when more than one 
degree of freedom was being controlled. 8 7 The poor reliability of bioampli- 
fiers has also been a major problem area. Summarizing, practical EMG control 
is beset with development problems. 

Of the many EMG-controlled prostheses, we describe Bottomley’s 
“myoelectric hand” 8 8 and a hand built by A. N. Skachkov 8 9 of the U.S.S.R. 

The Bottomley hand fits over the forearm of the amputee; two surface 
electrodes (Fig. 74) are fitted into the socket. To reduce crosstalk, signals 
from opposing forearm muscles are employed. The block diagram of the 
circuitry is illustrated in Fig. 75. Strain gauges in the hand yield a signal 
proportional to the force generated on an object by the hand; this signal in 
this local loop opposes the EMG signal and will cancel it out before the hand 
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Figure 74 The surface electrode used with the Bottomley EMG-controlled 
hand. 88 

can damage the object or itself by exerting too much force. Of course, the 
wearer can increase his grip upon an object by flexing his control muscles 
more; the motor will then operate until the strain-gauge signal again cancels 
out the EMG signal. 

The Skachkov hand is quite similar to that of Bottomley; however, the 
feedback device is different and rather interesting (Fig. 76). Instead of 
employing a local feedback loop, the wearer is brought into the loop by 
forcing him to monitor a vibratory feedback, which is proportional to the 
pressure on the thumb of the artificial hand. The purpose of the feedback is 
the same as that in the Bottomley hand. 

The subject of feedback to the wearer leads to the frequently suggested 
possibility of feeding back electrical signals directly into the operator’s 
muscles so that they will “know” intrinsically just what force the 
teleoperator is exerting. This would be an enviable kind of force feedback, 
but we have little knowledge as to how to put the idea into practice at the 
present time. The great hopes for EMG control stem from the philosophical 
observation that the human body is itself an electrical “machine” and so are 
many teleoperators. Should there not be many more intimate ways of 
coupling such similar equipments? 


HARDWARE AND SOFTWARE FOR 
SUPERVISORY CONTROL 

Control equipment in supervisory control consists of both hardware 
(typewriters, computer consoles, ( and devices like light pens) and software 
(computer programs, tapes, analog records, etc.). Supervisory control 
hardware and software are in the experimental stage today. 

Two groups have developed computer-controlled manipulators: 

L Case Western Reserve, where NASA and the AEC have sponsored work 
potentially leading to the semiautomatic disassembly of radioactive 
nuclear rocket engines. 7,90 




Figure 75 Block diagram of the control circuitry for Bottomley’s EMG- Controlled hand. 88 
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Figure 76 Block diagram of the Russian EMG-controlled prosthesis . 89 
1 1-Electrodes , 2-Amplifier , 3 -Discriminator, 4-Amplifier , 5-Motor , 6-Pressure 
transducer , 7-Amplifier , 8-Pulse generator , 9-Pulse shaper , 10-Amplifier , 
7 7 - Vibrator , 7 2-Summing circuit) 



Figure 77 Block diagram of the Case computer-controlled manipulator. The 
General Purpose Interface (GPI) described in the text bridges the interface 
between the computer and the manipulator . 90 

2. M.I.T. where general man-machine control problems are being attacked 
in both Sheridan’s group (NASA-DOD support 91 ), and Project MAC 
(DOD support 92 ). 

A typical system configuration of the Case computer controlled 
manipulator is illustrated in Fig. 77. The human operator can make inputs at 
two spots: the teletypewriter (TTY) and the conventional manual controls. 
The TTY, of course, is the input point for supervisory control instructions, 
which may be extremely simple incremental motion instructions or a 
command to carry out a complex subroutine of instructions stored in the 
memory.* 

Following the path of supervisory control information — horizontally 
from TTY to Manipulator in Fig. 77 — we see that the operator must first 
know what sort of instructions to type for the computer on the TTY; next, 


*A typical supervisory control subroutine employed in the Case system was 
described in Chap. 3. 
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the computer output must be converted into electrical voltage signals that 
will drive the manipulator joints the proper distances in the proper directions. 
The interface between the operator and computer is bridged by the Teletype 
Executive Control Program (TTY Exec); while the computer directs the 
manipulator through the General Purpose Interface (GPI) Unit. 

In the TTY Exec approach, all subroutines are initially stored in the 
computer’s memory. The operator must specify a basic positioning or 
path-control algorithm as well as specific distances and other data related to 
the desired manipulator action. The following example is from Beckett: 

Desired action: The manipulator hand should move 7.5 inches in a 

straight line along the direction in which the hand points. 

Actions: 

1. The operator types a V and the TTY Exec first calls the IDST 
subroutine (Fig. 78). 

2. The IDST subroutine types the letters DIST and then waits for the 
operator to specify the distance to be moved in inches. (Note: the 
IDST subroutine contains instructions that enable the computer to 
understand the specific distances typed by the operator.) 

3. The operator types 7.5, followed by a carriage return. 

4. The IDST subroutine stores the figure in a memory buffer. 

5. The TTY Exec (as instructed by the letter V typed first) now calls 
the VECTOR subroutine, which produces a series of signals that will 
drive the manipulator motors the required amounts — provided the 
signals are translated into motor voltages in the GPI. 



Figure 78 The teletypewriter is a common input device for computer- 
controlled manipulators. (Courtesy of Case Western Reserve.) 
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The General Purpose Interface (GPI) is too complex to be described here; 
the reader is referred to Taylor’s report. 90 The GPI provides a 
communication and control link between the computer and an arbitrary 
external machine (a manipulator here). It is designed to translate computer 
signals into basic manipulator control voltages. Address and function 
decoding are done within the GPI and it also provides status information lines 
to let the computer and operator know the current manipulator 
configuration. 

Summarizing the Case setup, the operator types abstract symbolic 
commands and specific data on the TTY. Using its programs and subroutines 
the computer converts the commands and data into a series of signals which, 
in turn, are converted by the GPI into the required manipulator voltages. 

Supervisory control reaches its ultimate development in the fully 
developed robot; that is, a machine that carries out high-level, abstract 
commands without human assistance at any primitive levels of action. The 
robot developed by Stanford Research Institute (SRI) under DOD 
sponsorship 1 7 is not a teleoperator per se because it manipulates nothing and 
rolls rather than walks (pedipulates) (Fig. 79). Nevertheless, arms and hands 
could be added and operated within the general hardware/software 
framework. In actuality, the SRI robot is a harbinger of future teleoperator 
technology. 

The SRI scientists suggest that their robot (or any robot/teleoperator) 
will eventually be able to operate at four levels of control: 

1. The immediate-action level, where the operator directly activates the 
motors and sensors. (This mode of operation is equivalent to the 
operation of unilateral manipulators by a switchbox. In other words, 
no supervisory control exists.) 

2. The tactical level, where the robot solves simple problems in navigation 
and locomotion without the help of the operator. (The VECTOR 
subroutine employed by the Case computer-controlled manipulator 
falls in this category.) 

3. The strategic level, where the robot finds specified objects and 
relocates them. (The Case and M.I.T. computer-controlled manipula- 
tors can carry out supervisory instructions of this type.) 

4. The problem-analysis level, where the robot translates a high-level 
command into a series of subtasks according to some criterion of 
performance. 

The SRI robot is controlled through a teletypewriter, just as the Case 
manipulator (Fig. 80). There is also an analogous computer plus its software 
(programs and subroutines). The computer-robot interface is intimate and 
specialized, not general-purpose like the Case GPI. While the Case system has 
status indicators, it does not have the full array of kinesthetic sensors 
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Figure 79 The SRI robot. While this robot does not have the arms and legs 
needed to qualify as a teleoperator , its autonomous functions may eventually 
be incorporated into teleoperators. (Courtesy of C. Rosen, Stanford Research 
Institute.) 
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possessed by the SRI robot. The SRI robot uses both the visual and 
kinesthetic/eedback in local control loops. 

Using visual feedback through its TV and kinesthetic data (obtained from 
bumping objects in its environment) the robot can construct a model of its 
environment. The model includes its own location as well as the positions, 



j VEHICLE_ _ _ J 

MOTION 

Figure 80 Control plan for the SRI robot. Note the local control loops. 
DOJOURNEY is a specific robot program . 1 7 

orientations, and in some cases identities of the objects. The robot can 
reconnoiter its surroundings itself — a valuable property for a teleoperator 
operating, say, beneath the sea, where feedback to the operator may be 
sparse. In most computer-controlled manipulators, obstacles to be avoided 
must have their coordinates placed in the computer’s memory by the 
operator; not so with the SRI robot. 

A number of lower-level subroutines have already been successfully tested 
with the SRI robot. A typical immediate-action-level command is CALL 
MOVE (N, HANDLE). This command moves the vehicle forward (+) or 
backwards (— ) N 32nds of an inch. A bump can interrupt this subroutine, in 
which case HANDLE (a sort of register) will be set at the number of motion 
increments not actually completed. Like all other computer-controlled 
devices discussed in this book, the SRI robot moves in a quantized fashion. 

The GOTO executive subroutines operate at a somewhat higher level. 
They take the robot from one point to another, using its model of the 
environment to help it avoid obstacles. More complex programs are being 
written for the robot, such as that described in the flow chart of Fig. 81. 
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Figure 81 Flow diagram for a more advanced program proposed for the SRI 
robot. A similar program could be used by a teleoperator searching for a 
specific object . 1 7 


Clearly, by building upon a foundation- of simple autonomous (primitive) 
functions and utilizing its feedback data, the SRI robot and its descendants 
will be able to carry out more and more generalized, higher-level commands 
from the human operator. As stressed frequently in earlier chapters, 
teleoperators are also following this path toward greater autonomy. 









Chapter 6 


DISPLAYS 

TELEOPERATOR DISPLAYS 

In the broadest sense, a tele operator display is the output station for all 
sensory information fed back to the operator. The display is the output 
counterpart of the input hardware described in the last chapter. Together, 
controls and displays connect man to machine and vice versa; they are 
interface devices (Fig. 8). 

The word “display” connotes a pictorial, visual view of some scene or 
situation. Control engineers broaden the meaning to include abstract and 
symbolic displays, which represent scenes or situations in less natural terms, 
such as a digital distance reading or a stylized manipulator configuration. In 
teleoperator engineering, the concept of a display must be expanded to 
include the complete panorama of man’s senses; past, present, and predicted 
future; couched in anthropomorphic or abstract language. A TV scene of the 
interior of a hot cell is a display; but so is the force feedback in the arm of an 
electric masterslave; so is a warning buzzer signaling that a joint’s limit of 
travel has been reached. 

Display design is a field of great importance in the engineering of aircraft, 
manned spacecraft, and submarines, where the operator must be aware of a 
great deal more than he can perceive looking through a window or porthole. 
In fact, windows and portholes are not used at all on some vehicles; instead, a 
“picture” of the environment is drawn by radars, sonars, and other sensors. 

Teleoperators are manipulatory and sometimes pedipulatory and mobile; 
vision is crucial to good performance in most cases; force feedback and 
tactual feedback are desirable where they can be obtained at a reasonable 
price. The other senses, such as sound, are not nearly as important. 
Teleoperator displays in use today differ little from those in advanced 
aerospace and undersea vehicles. In fact, they owe much to the display theory 
and hardware developed for these vehicles. 9 3,94 

Ideally, a tele operator display would show the environment of interest 
(including the objects to be manipulated, local temperature, and other 
factors) and the present position or status of the teleoperator. This type of 
information gives the operator a seat-of-the-pants feel for the situation.* 


*Teleoperator technology will eventually be able to supply multiple operator 
feedback terminals so that many scientists could participate in, say, remote lunar 
exploration; although only one person would be the true operator, of course. 
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Good displays project the operator into the place where his ersatz hands and 
legs are working — he identifies with the situation. We have stressed in earlier 
chapters that manipulation also requires planning and strategy formulation. 
We therefore must make room in our definition of displays for portraying 
executive information that will help the operator make decisions. Two special 
types of displays that fall in this category are the predictor and historical 
displays; one looks into the future using known physical laws; the other 
searches the past for relevant information. Teleoperator displays can and 
probably will be much more generalized than the hot-cell windows and 
closed-circuit TV typical of most extant tele operator applications. 

Display engineering has been largely intuitive in the past. In the case of 
teleoperators, the basic tenet has been to make the display as real as possible ; 
that is, to duplicate the sight, sound, and feel of the task as faithfully as 
possible. This philosophy is a natural corollary to the assertion that 
teleoperator controls should be as anthropomorphic as possible. Both of these 
views are being challenged today. 

While no formal tele operator display theory exists, some progress has 
been made recently in formalizing display theory for use in conventional 
manual control situations; i.e., aircraft and undersea craft. Kelley’s book 27 
and a recent paper by McRuer and Jex 94 are representative of this work. 
Most display theory deals with forced-input tracking situations and offers 
little to the designer of a tele operator display. 

Conventional display theory does offer a checklist of points to consider 
and pitfalls to avoid in tele operator display design: 

1. Noise seriously degrades displays. A reasonable signal-to-noise ratio 
must be obtained in all sensory dimensions. 

2. The effectiveness of a display is reduced by intermittence ; that is, the 
reduction in time intervals when the display is active or sensed by the 
operator. 95 This factor applies to the time-multiplexing of display 
information and the sample d-data aspects of the operator as he shifts 
his attention from one display to another. 

3. Time-delayed feedback is highly disruptive as mentioned in Chap. 3. 
Predictor displays may minimize this effect. 

4. Visual display parameters of magnification, framing, color, dimen- 
sionality, contrast, brightness, etc. must be considered, 96 although few 
objective data are available to guide the designer. 

The human factors scientist obviously has a great deal of experimentation 
ahead before firm guidelines emerge for teleoperator display engineers. The 
make-it-anthropomorphic school is supported by intuition, but intuition has 
been proven false in the past. The fact is we really know very little about 
engineering teleoperator displays. 
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NATURAL VISUAL DISPLAYS 

Although a surprising amount of manipulation can be carried out by the 
sense of feel alone — blind workers, for example — sight is by far the most 
important human input channel. In some manipulator and pedipulator tasks, 
the operator sees his work and its environment directly or perhaps through 
windows, such as those in hot cells and submersibles. Calling a hot-cell 
window a display may stretch the definition a bit; but the hot-cell window is 
a processor of optical data. A hot-cell periscope, or fiber optics device, which 
provides the manipulator operator an indirect view of his work, fits the 
definition of display even better. A periscope processes optical information, 
often magnifying the image, and presents the hot-cell interior to the human 
eye at its focal plane. 

We term direct and indirect viewing systems “natural” because there is no 
intentional distortion of the scene. In natural visual displays, the object is to 
make the view presented to the operator as realistic as possible. 

Another natural viewing system is closed-circuit television. In TV, the 
scene is disassembled and then reconstituted electronically at the display. In 
between, there may be considerable data processing, particularly in those 
television systems employed in astronautical ventures. Despite these 
opportunities, the guiding philosophy for most TV displays is “make it 
natural.” In the backs of our minds, however, we should remember that in 
TV systems resides the potentiality for creating unnatural, perhaps symbolic, 
displays. To illustrate, signals from a TV camera might be combined with 
other sensory data, say, manipulator joint position data, to draw an abstract 
three-dimensional picture of the scene, somewhat after the fashion of 
air-traffic-control displays. 

With this introduction, the reader should turn to Ref. 1, Chap. 6, for a 
discussion of the engineering features of natural visual displays and the 
sensors associated with them. The remainder of this chapter deals with 
symbolic visual displays and displays involving senses other than sight. 
Reference 1 dealt with teleoperator sensors and displays in use today; our 
present purpose is to introduce some thoughts about tele operator displays of 
the future. 

SYMBOLIC AND ABSTRACT VISUAL DISPLAYS 

Once it is admitted that natural, pictorial displays convey only part of the 
information an operator desires, the way is open to symbolic displays. The 
word “symbolic” is used here to mean non-pictorial. A simple warning light 
indicating that a manipulator limit of motion has been reached for a 
manipulator is a symbolic display because an “on” light is a code signal 
understandable to the operator — a signal conveying far more than one bit of 
information. 
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The basic function of a display is to provide the operator with enough 
information to make decisions; this information need not be pictorial to be 
useful/ In fact, manipulation can be accomplished without natural, pictorial 
feedback at all. Computer-controlled manipulators never “see” their targets at 
all. Conceivably, a human operator could manipulate objects given enough 
force and tactual feedback plus a good repertoire of executive signals, 
although performance might suffer considerably without vision. 

Besides warning lights and other status signals, what other kinds of 
symbolic visual displays might be useful in tele operator work? Perhaps the 
most obvious type would be an abstract portrayal of the working 
environment, its targets, and the tele operator arms and hands — a substitute 
for a natural view, which might be unobtainable. The scene could be drawn 
on a cathode ray oscilloscope tube (CRT) in stylized fashion, showing the 
manipulator and its targets vividly in three dimensions, possibly color-coded 
for easy identification, noise could be suppressed, and target data could be 
inserted verbally near the target image on the CRT (the air-traffic-control 
example again) (Fig. 82). The Computer Image Corporation has been 



Figure 82 A possible abstract visual display indicating the configuration of a 
teleoperator and the external environment. Such a display need not be drawn 
from visual data alone; Le., sonar , radar, and status sensors can be employed. 

pioneering this type display. Such an abstract, coded representation might be 
much easier to work with than a natural view of the scene. Furthermore, this 
kind of display could be drawn from many different kinds of sensory inputs: 
iconoscope, radar, sonar, and, of course, status data. There would certainly be 
anthropomorphic aspects to an abstract display in terms of spatial 
correspondence, but we have no experimental assurance that 
anthropomorphism is required. 

Few symbolic or abstract teleoperator displays have been built yet, so we 
continue primarily in a prophetic vein, buttressed by some anticipatory work 
done here and there for other applications. 

Symbolic displays are part and parcel of everyday life; viz., fuel gauges 
and speedometers in automobiles. A symbolic approach to manipulation is 
not hard to imagine, though there is no proof that it would be effective. Most 
manipulatory tasks can be described in terms of seven dimensions; we might 
build a symbolic display along these lines (Fig. 83). The three degrees of 
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Figure 83 A possible abstract visual display indicating the status or 
configuration of a seven-degree-of- freedom teleoperator . 

freedom representing the position of the hand relative to the target could be 
portrayed on a CRT-drawn set of Cartesian axes. Similarly, the hand 
orientation — three more degrees of freedom — could be displayed as a vector 
relative to the orientation of the object. Finally, hand closure around the 
target, the seventh degree of freedom, could be represented by a vise-like 
sketch. The grip in pounds could be displayed numerically next to the grip 
display. 

Why would one want to employ symbolic or abstract displays instead of 
honest, natural pictorial displays? 

L The natural visual display may possess noise, distortion, and bad 
contrast. Signal processing can clean it up. 

2. A natural visual display requires an immense quantity of 
information — a large bandwidth. Symbolic and abstract displays can 
be drawn with far less bandwidth. On a lunar spacecraft, for example, 
signal processing equipment can eliminate all data in the natural scene 
except those pertaining to the targets and the tele operator 
configuration. 

3. In some instances, there is no natural visual display because natural and 
artificial lighting are absent. 
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4. Symbolic and abstract displays may lead to better performance of 
teleoperators. (A contentious statement.) 

An extreme example of abstract, symbolic displays is the teletypewriter 
employed in supervisory control. An output device as well as an input device, 
the teletypewriter can print out manipulator configuration coordinates, the 
geometric relationship of the targets, and status data-in fact, anything we 
wish to know about the teleoperator and its task. Of course, there is no 
anthropomorphism in the printout of a teletypewriter; it is hard to imagine 
how an operator might identify himself with the task. Operation would 
certainly not be natural in the sense of everyday experience. Some people, 
however, identify well with symbols and mathematical relationships. A 
matrix is as real to them as an actual force on the target. Manipulation in this 
case would be much like playing chess without a chessboard — some people 
can do it. 

The hardware available and under development for the display of abstract 
and symbolic information, like the television systems used for natural visual 
displays, is beyond the scope of the survey. The variety of media is large and 
gives the teleoperator designer ample opportunity to explore new modes of 
machine-to-man communication. We list some types of visual displays: 


Thermochromic 

Fluidic 

Electrostatic 

Plasma 


Photochromic 

Magnetic 

Laser 

Electroluminescent 


Reference 97 surveys the state of the art for these types. The more 
conventional CRT and projected large-screen displays are discussed in Refs. 
98 and 99. 

Some of the experimental displays under development for submarines are 
similar to the displays we might expect in a teleoperator. McLane and Wolf 
have summarized some of the Navy’s symbolic display work. 1 00 In Fig. 84, a 
type of symbolic depth-azimuth display is portrayed. Note that sonar supplies 
the basic environmental information; this is codified into abstract symbols 
representing obstacles and targets. The depth and azimuth scales allow the 
operator to avoid obstacles and, perhaps, torpedoes. Ship status is also 
indicated. Horizontal range to the target is not given on this display, but it 
could be indicated numerically on the scope face. Another type of symbolic 
visual display is shown in Fig. 85. On this the ship’s attitude is portrayed 
geometrically and distance to the target or obstacle can be judged from the 
perspective present in the display. Depth relative to the target is also intrinsic 
in the display. One can imagine displays similar to these drawn for a 
manipulatory situation — Fig. 82 could be redrawn with perspective and 
other types of coding added. The geometrical cues relating ship to obstacle 
are analogous to those relating a manipulator arm to a target. 
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• TRACKING SYMBOL + SHIPS Z, ^ 

(Non-Linear Quickening) 

Figure 84 A possible type of symbolic visual display showing a submarine's 
depth-azimuth relationship to an external target. The target and relative 
configuration of a teleoperator could be indicated by a similar display} 00 



!\ TRACKING SYMBOL SHIP'S Z. \ p GIVEN BY TOTAL WINDOW 

(Perspective Tracking Symbol) VIEW. GRIDS, PATH. HORIZON 


Figure 85 A possible type of abstract visual display indicating a submarine's 
attitude and relationship to an external target} 00 
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One type of abstract visual display that has been developed in connection 
with teleoperator technology is the M.I.T. tactual sensor. One version of this 
sensor was described in Ref. 1. A later model is pictured in Fig. 86. Touch in 
this sensor is detected as a distortion of a flexible surface, which is then 
imaged upon a fiber-optics bundle and ultimately on a CRT. We have in effect 
an abstract visual display of touch as “felt” by the manipulator jaws (Fig. 
87). Despite the fact that there is some correlation between the shape of an 
object and the pattern viewed on the CRT, correct and facile pattern 
interpretation requires operator training and may divert him from the task. 


VISUAL PREDICTOR DISPLAYS 

Displays which help the operator predict the future are helpful in 
dynamic situations, such as high-speed piloted aircraft and submarines. 
Manipulators ordinarily move so slowly that predictor displays are of little 
importance. The major exception occurs where significant time delay exists. 
(See Chap. 3 for the effects of time delay.) In cislunar space, on the Moon, 



Figure 86 Schematic of the M.I.T. optical touch sensor . See Fig. 87 for 
typical displays . 1 3 



hexagon 


circular cross section 
with concentric hole 


Figure 87 CRT displays showing the effects of various shapes on the M.I.T. 
tactual sensor. (Courtesy of T. B. Sheridan, M.I.T.) 

and beyond, teleoperator performance can be enhanced if the operator back 
on Earth has some sort of predictor display that estimates the consequences 
of his actions before he issues commands to the Earth-based transmitter. 

Predictor instruments look ahead in time by constructing “models” of the 
situation — primarily models of the machine and its environment. The model, 
possibly an electrical analog, is then run faster than real time (that is, ahead 
of real time) and its performance is displayed for the operator. In many 
human tasks, a person performs these computations in his head intuitively. In 
guiding an automobile around a curve, the driver projects his vehicle’s 
position as a function of time for various combinations of control actions. 
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The model of the situation employed by a predictor instrument is usually 
displayed visually. However, there is no reason why force and tactual 
feedback cannot be predicted for the operator. In fact, if predicted force 
feedback could be added to predicted visual feedback on the same time scale, 
the operator would have excellent grounds for decision-making. 

A visual predictor display must be abstract or symbolic because there is 
no knowledge of the real natural world of the future — -only projections. It is 
customary, however, to display projections in time in anthropomorphic 
fashion, say, as a projected vehicle track on the actual televised scene. 

Aircraft instrument panels have long utilized time-derivative (rate of 
change) data in helping the pilot maneuver his craft.* Ziebolz and Paynter 
discussed the possibility of improving upon simple derivative information by 
employing fast models or analogs of the entire system. 26 In the early 1960s, 
Kelley and his associates developed a Predictor Instrument for the Navy to 
help control submarines. 101,102 These ideas form the basis for tele operator 
predictor displays. 

Because of its historical importance, we sketch a few details of Kelley’s 
Predictor Instrument. Figure 88 shows the block diagram for this device. The 
heart of the Predictor Instrument is a miniature computer (an analog 
computer in this case) that models the system. As the sensors feed in 
information about the present, it predicts the future for various “degrees of 
freedom” (Fig. 89). The operator “sees” the future as a function of time and 
takes whatever action seems appropriate. The original purpose of the 
Predictor Instrument was not to overcome signal transmission time delay but 
rather to offset the operator’s reaction time and warn him of future 
consequences that he might not anticipate from real-time data alone. 

NASA and the Air Force have investigated predictor displays for use in 
orbital rendezvous, an operation where terrestrial vehicle experience is not 
too helpful. 1 03-1 05 These studies employ fast-time models for prediction; 
again the objective is to help the operator in a complex real-time situation. 
Air Force-sponsored simulator studies confirm that a predictor display 
materially helps the astronaut. 

More germane to the teleoperator time-delay problem in outer space are 
the studies of predictor displays for lunar vehicles. Dunlap and Associates, 
Stanford University, General Motors Corp. and others have completed studies 
and simulation experiments. 21,106 Again, a fast-time model of the physical 
system constitutes the basis for prediction. 

The studies proposed superimposing a symbolic track representing the 
predicted vehicle motion upon a symbolic or pictorial display of the lunar 
environment (Fig. 90). It should be emphasized that these proposed displays 
are the results of studies and that no hardware exists at the present time. 


*The use of derivative information in generating displays is termed “quickening” and 
the quickened display is often called “augmented.” 
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Figure 88 General block diagram of the Predictor Instrument. (Courtesy of 
C. R. Kelley, Dunlap and Associates.) 
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Figure 89 The Predictor Instrument developed by Kelley and colleagues at 
Dunlap and Associates , gives the operator a preview of the future using a fast 
model of the physical system . This particular display forecasts depth of a 
submarine. 


Leslie et al . 22 have carried out simulator studies of the effectiveness of 
predictor displays in the operation of lunar roving vehicles. Their results 
indicate that the deterioration of performance due to time delay can be all 
but erased through the use of a predictor display. 
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Figure 90 A possible type of predictor display for a lunar roving vehicle. 

The only tele operators to operate under time-delay restrictions to date 
have been the Surveyor surface samplers (see Chap. 5). Operations with the 
surface sampler were slow and deliberate and made use of the move-and-wait 
strategy. The primary display aiding surface-sampler manipulation of the 
lunar soil was pictorial, using the pictures taken by the Surveyor vidicon 
camera. Because of the 1 .3-second signal-propagation time delay and the time 
required to scan and transmit the vidicon image, the display was what we 
might call “historical” in nature. Each picture was several seconds old at best. 
The operator of the sampler could, of course, examine as many of these still 
photos as he wished, but they gave him little identification with the dynamics 
of the experiments. 

Movies made after the mission from successive Surveyor pictures have 
added real-time dynamic insight to the sampler operations. By showing 
several hour’s pictures in a few seconds, the motion of the sampler and soil 
movement can be seen. In effect, the human brain melds the time-separated 
photos into a smooth whole. In future lunar operations, sped-up historical 
displays may quickly recapitulate the last hour of motion to lend reality to 
the present scene. In planning his next move, the operator could command 
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this review of past operations at will; his brain could then project 
consequences of his actions better. Time delay is not eliminated, of course, 
but time seems compressed to terrestrial scale and the operator can use his 
worldly experience to predict what might happen for each prospective 
command. 


FORCE FEEDBACK AND TACTUAL DISPLAYS 

Next to vision, force feedback to the hands, arms, or legs of a 
teleoperator is the most important type of “displayed” information. The 
mechanical and electromechanical bases for force feedback were sketched 
earlier in this book and in Ref. 1. Cables and servo motors force slave arms to 
follow master arms and vice versa when the slave arm encounters an object. 
The “display,” of course, is the totality of forces and pressures applied to the 
hands, arms, and legs of the operator. 

In tactual (or tactile) feedback, the situation is complicated by the fact 
that a well-defined, two-dimensional field of pressure stimuli is desired. 
Touch feedback devices thus take on some of the features of visual displays. 
As mentioned earlier, some tactual displays are actually visual in character. 
Here, we deal only with those displays that stimulate the surfaces of the 
fingers, although it might be argued that vibrator alarms, such as those 
associated with artificial limbs (see Chap. 5) might also be considered useful 
tactual displays for teleoperators. 

Bliss and his colleagues at Stanford Research Institute (S.R.I.) have 
developed tactual displays for a wide variety of potential applica- 
tions. 1 07,1 08 Bliss’ group has constructed reading machines for the blind 
which convert printed letters into tactual displays that can be read by the 
fingers. If the fingers can discern the shapes of the letters from a tactual 
display, the same displays could give an operator a good sense of feel in 
remote manipulation. 

Bliss and his associates have worked with electromechanical, piezoelectric, 
electrical, and air-jet stimulators. The air-jet stimulators have proven 
successful and have been employed in many S.R.I. experiments (Fig. 91). 
Airjet stimulators arrayed 12x12 have been built finger-tip size — this is the 
array that resolves letters of the alphabet. One can conceive of such arrays 
being built into the master hand controls of advanced master-slaves, with each 
of the 144 stimulators actuated by a corresponding pressure-sensitive spot 
(perhaps a piezoelectric crystal) on the slave hand. 

How useful would tactual displays be if visual displays and force feedback 
were already applied to a given problem? Intuitively, one would say tactual 
feedback must be beneficial; but no one knows for sure. Bliss’ human factors 
studies with tactual arrays have indicated that the human delay time with 
tactual displays alone is appreciably longer than for an equivalent visual 
display alone. However, human reaction time when visual and tactual displays 
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Figure 91 A 12 X 8 array of air-jet tactual stimulators . The active area of the 
array is finger-tip size, (Courtesy J. C. Bliss , Stanford Research Institute.) 


are used simultaneously is shorter than for either display alone. Some 
manipulatory experiments will have to be made to determine the true utility 
of tactual displays. Any performance advantages would have to be weighed 
against the increased complexity of the teleoperator system and the 
engineering difficulty of installing the sensors and stimulators on machine and 


man, 



Chapter 7 


CONTRIBUTIONS FROM TELEOPERATOR 
CONTROL TECHNOLOGY 


Teleoperator control technology is young compared to the experience we 
have amassed in the manual control of aircraft and other vehicles. Mostly, the 
flow of control technology has been into rather than out of the tele operator 
field. Servos, joystick controls, and computer software came well-developed 
and were integrated early into tele operator technology essentially as is. 
Despite their relative youth, teleoperators have contributed some significant 
theory and hardware developments that will eventually find applications 
elsewhere in our technological repertoire. Below, we tabulate these key 
contributions.* First the important theoretical developments. 

Possible Applications to 

Contribution Other Areas 


Tele operator state theory 
(M.I.T., U.C.L.A.) 

Human factors studies of 
man and teleoperator and the 
effectiveness of displays 
(U.C.L.A., Electric Boat, Stan- 
ford Research Institute 
(S.R.I.), U.S.A.F. (Wright 
Field) 

Studies of the effects of time 
delay (M.I.T., Stanford 
University) 

Development of a rationale 
for supervisory control 
(M.I.T.) 

Programs and software for 
computer-controlle d 
manipulators (M.I.T., Case) 


Automated factories, robots, almost any 
machine 

Better understanding of man-machine 
relationship useful in the design of all 
manually controlled machines 


Applicable to the operation of all very 
remote machines (planetary probes, 
automated life- detection laboratories) 
All manually controlled machines use 
these concepts (aircraft, submarines, 
etc.) 

Almost any man-machine combination, 
(automated factories, robots, aircraft) 


*See Ref. 1, Chap. 2, for a thorough review of teleoperator applications. Only 
specific contributions from teleoperator control technology are covered here. 


139 



140 


TELEOPERATOR CONTROLS 


Computer-assisted adaptively Automated factories and robots 
controlled machines (S.R.I., 

m.i:t.) 

Multijoint servo theory (GE) All servoed machinery 

In addition to the above theoretical developments, some important pieces 
of tele operator hardware have been built that may have application in other 
areas. 


Contribution 


Possible Application to Other Areas 


The ANL E4A electric 
master-slave 


The ANL head-controlled 
TV display 

Tactual displays 
(S.R.L, M.I.T.) 


The GE Walking Truck 
and Hardiman 


This control technology could be used in other 
remote-control applications (undersea 
mining, spacecraft control, unmanned 
weather stations, etc.) 

All remote operations whether teleoperators 
are employed or not. (Remote surveillance 
and exploration) 

In aids for the blind, possibly in oil-well work 
and deep sea exploration where a sense of 
touch is helpful in installing equipment and 
instruments 

The electrohydraulic and high-power electric 
servos developed here and their controls 
should also be useful in earth-moving 
equipment, mining machinery, etc. 


Summarizing, developments in tele operator control are helping 
significantly in the automation of many human tasks and the successful 
bridging of man-machine interfaces. These are two distinctly different areas. 
The first brings to mind visions of highly automated factories and household 
robots. This sort of machinery is nearly fully automatic and cannot be 
classified with teleoperators; however, teleoperator control, particularly 
computer-aided supervisory control, is definitely oriented in the direction of 
automatic control. Teleoperator control is philosophically somewhere 
between the disciplines of manual control and automatic control; and it 
draws from and contributes to each. The second important application 
area — bridging the man-machine interface — includes all man-machine 
systems and in the modern world, this means everything from computers to 
spacecraft to the telephone system. All advances in teleoperator control 
technology enhance our ability to weld man successfully with other kinds of 
machines. 

In teleoperator control, we stretch man to make him more like a machine; 
viz., nonanthropomorphic control. We also mould machines so they are more 
like men; viz., exoskeletons, artificial limbs, and manipulators. It is because 
teleoperators help hybridize man and machine that they are so important to 
our basic technology. 
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